Massage teaching method and system based on multi-modal data collaboration

Abstract

The invention relates to a massage teaching method and system based on multi-modal data collaboration, and belongs to the technical field of massage teaching. The method comprises the following steps: acquiring pressure distribution data and kinematics data in a massage process of a student to obtain pressure parameters and motion parameters including whole palm motion parameters and local fingertip motion parameters; through a pre-trained action correlation model, palm overall motion parameters with coordinated actions and fingertip local motion parameters with coordinated actions are screened out and fused with pressure parameters, and dual-mode fusion features including the action synchronization rate of the palm and the fingertips and the force-speed synchronization rate of the pressure and the speed are obtained. And obtaining a current massage manipulation type according to the dual-mode fusion features, the pressure parameters, the palm overall motion parameters with coordinated actions and the fingertip local motion parameters with coordinated actions, and calling matched standard massage parameters to calculate a difference degree so as to generate a current massage score and personalized massage teaching guidance suggestions. Therefore, the massage teaching efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of massage teaching, and in particular relates to a massage teaching method and system based on multi-modal data cooperation. BACKGROUND

[0002] The existing massage technique teaching has the following two major problems: (1) The teaching personnel mainly rely on oral and mental instruction, and there is a lack of quantitative standards for key parameters of massage techniques, so that students cannot accurately perceive and reproduce standard techniques, and the teaching personnel can only evaluate the students' operation through visual observation, which is highly subjective and has large individual differences, and cannot form objective and unified evaluation basis, resulting in uneven training effect of students; (2) The existing few auxiliary teaching devices mostly use single type sensors, which cannot comprehensively capture the multi-dimensional characteristics of massage techniques, and cannot analyze the defects in students' operation from multiple dimensions, and cannot provide precise error correction guidance for students, resulting in low teaching efficiency.

[0003] In order to solve the above massage teaching difficulties, it is urgent to develop a teaching technology capable of realizing multi-modal parameter cooperation collection, objective quantitative evaluation and precise guidance of massage techniques. SUMMARY

[0004] The technical problem solved by the present application is to provide a massage teaching method and system based on multi-modal data cooperation, which realizes multi-modal parameter cooperation collection of massage techniques, objective quantitative evaluation and precise guidance of students' operation, and improves the teaching efficiency.

[0005] In order to solve the above technical problems, the technical scheme adopted by the present application is: In a first aspect, the present application provides a massage teaching method based on multi-modal data cooperation, comprising: real-time collection of pressure distribution data and kinematics data in the massage process of students, pre-processing of the pressure distribution data and the kinematics data respectively, obtaining pre-processed pressure distribution data and pre-processed kinematics data, and pressure feature extraction on the pre-processed pressure distribution data to obtain pressure parameters, while motion feature extraction on the pre-processed kinematics data to obtain motion parameters, the kinematics data including palm overall motion data and fingertip local motion data, the pressure parameters including palm overall pressure parameters and fingertip local pressure parameters, and the motion parameters including palm overall motion parameters and fingertip local motion parameters; obtain a current acquisition timestamp, time slice the palm overall motion parameter and the fingertip local motion parameter according to the current acquisition timestamp, obtain the palm overall motion parameter and the fingertip local motion parameter under all time slices, input the palm overall motion parameter and the fingertip local motion parameter under each time slice into the pre-trained action correlation model one by one to calculate the action correlation strength, obtain the corresponding action correlation strength under all time slices, and screen out the action-coordinated palm overall motion parameter and the action-coordinated fingertip local motion parameter according to the corresponding action correlation strength under all time slices; fuse the action-coordinated palm overall motion parameter and the action-coordinated fingertip local motion parameter with the pressure parameter to obtain a dual-mode fusion feature, the dual-mode fusion feature including an action synchronization rate of the palm and the fingertip and a force-velocity synchronization rate of the pressure and the speed; input the dual-mode fusion feature, the pressure parameter, the action-coordinated palm overall motion parameter and the action-coordinated fingertip local motion parameter into the pre-trained massage technique recognition model to recognize the massage technique, obtain a current massage technique type, call matched standard massage parameters from a pre-constructed standard technique library according to the current massage technique type, calculate a difference degree of the standard massage parameters and the current massage parameters, generate a current massage score according to the difference degree, and generate an individualized massage teaching guidance suggestion according to the current massage score.

[0006] The beneficial effects of the present application are that the dual-mode pressure distribution data and kinematic data are synchronously and real-time collected, the one-sidedness of the traditional single-mode collection is overcome, the corresponding pressure features and motion features are extracted from the pressure distribution data and kinematic data, the obtained pressure parameters include the overall palm pressure parameters and the local fingertip pressure parameters, the motion parameters include the overall palm motion parameters and the local fingertip motion parameters, the pressure parameters and motion parameters are split from the two dimensions of the overall palm and the local fingertip, and the accuracy and comprehensiveness of the obtained pressure parameters and motion parameters are improved. The overall palm motion parameters and the local fingertip motion parameters are time-sliced, the overall palm motion parameters and the local fingertip motion parameters under each time slice are input into the pre-trained action correlation model one by one for action correlation strength calculation, the accuracy of the calculated action correlation strength is improved, and thus the accuracy of the screened action-coordinated overall palm motion parameters and the action-coordinated local fingertip motion parameters is ensured. When the massage manipulation is identified, not only the pressure parameters, the action-coordinated overall palm motion parameters and the action-coordinated local fingertip motion parameters are considered, but also the dual-mode fusion features of the action synchronization rate of the palm and the fingertip and the force-velocity synchronization rate of the pressure and the speed are considered, the one-sidedness of only focusing on the single mode and failing to reflect the cooperative relationship of the dual mode is overcome, the accuracy of the massage manipulation identification is improved, and thus the accuracy of the generated current massage score is ensured, the operation of the student can be objectively quantitatively evaluated and accurately guided, and the teaching efficiency is improved.

[0007] Optionally, the preprocessing includes noise reduction processing, the pressure distribution data and the kinematic data are respectively preprocessed to obtain preprocessed pressure distribution data and preprocessed kinematic data, the preprocessed pressure distribution data is subjected to pressure feature extraction to obtain pressure parameters, and the preprocessed kinematic data is subjected to motion feature extraction to obtain motion parameters, and the pressure parameters include: The pressure distribution data is subjected to noise reduction processing by a median filtering method and a Gaussian smoothing method respectively to obtain noise reduction processed pressure distribution data, and the noise reduction processed pressure distribution data is divided into palm region pressure distribution data and local fingertip pressure distribution data based on a preset hand contact region division rule; The palm region pressure distribution data and the local fingertip pressure distribution data are respectively converted into two-dimensional pressure thermal maps by a bilinear interpolation method to obtain a palm overall two-dimensional pressure thermal map and a local fingertip two-dimensional pressure distribution map, and the palm overall two-dimensional pressure thermal map and the local fingertip two-dimensional pressure distribution map are respectively reconstructed into three-dimensional pressure thermal maps by a voxel reconstruction technology to obtain a palm overall three-dimensional pressure thermal map and a local fingertip three-dimensional pressure thermal map, and the palm overall three-dimensional pressure thermal map and the local fingertip three-dimensional pressure thermal map are respectively subjected to pressure feature extraction to obtain palm overall pressure parameters and local fingertip pressure parameters; The palm overall motion data is denoised by a Kalman filtering method to obtain denoised palm overall motion data, and the fingertip local motion data is denoised by a particle filtering method to obtain denoised fingertip local motion data; motion feature extraction is performed on the denoised palm overall motion data and the denoised fingertip local motion data to obtain corresponding palm overall motion parameters and fingertip local motion parameters.

[0008] According to the above description, the median filtering method and the Gaussian smoothing method are used for denoising the pressure distribution data, which can accurately remove environmental interference in the collection process and improve the quality of the pressure distribution data. The denoised pressure distribution data is divided into palm region pressure distribution data and fingertip local pressure distribution data, and pressure feature extraction is performed using corresponding three-dimensional pressure thermographs, which can accurately capture the pressure parameters of the palm overall and the fingertip local. For kinematic data, the Kalman filtering method suitable for smooth motion is used for denoising the palm overall motion data to improve the smoothness of the palm overall motion trajectory, and the particle filtering method suitable for fine motion is used for denoising the fingertip local motion data, which can effectively preserve small detail data and improve the quality of the denoised fingertip local motion data, thereby ensuring the accuracy of the obtained palm overall motion parameters and fingertip local motion parameters.

[0009] Optionally, the palm overall motion parameters and the fingertip local motion parameters are time-sliced according to the current collection timestamp to obtain palm overall motion parameters and fingertip local motion parameters under all time slices, and the palm overall motion parameters and the fingertip local motion parameters under each time slice are input into a pre-trained action association model one by one to calculate the action association strength, to obtain corresponding action association strengths under all time slices, including: The motion cycle of the palm overall motion corresponding to the preprocessed palm overall motion data is taken as a time reference to divide the palm overall motion into overall stages to obtain an overall stage time sequence, and the fingertip local motion corresponding to the preprocessed fingertip local motion data is divided into local stages based on the fingertip local motion parameters to obtain a local stage time sequence; the local stage time sequence and the overall stage time sequence are associated and mapped to construct a stage association mapping rule of the palm overall motion and the fingertip local motion. According to the stage association mapping rule and the current collection timestamp, the palm overall motion parameters and the fingertip local motion parameters are time-sliced according to a preset time slice to obtain palm overall motion parameters and fingertip local motion parameters under all time slices. a hand back center coordinate in the matched palm overall motion parameter under each time slice is taken as an original coordinate to establish a hand three-dimensional coordinate system, and a fingertip end coordinate in the corresponding fingertip local motion parameter is converted into a relative coordinate based on the hand three-dimensional coordinate system, so as to realize spatial calibration of the matched palm overall motion parameter and the fingertip local motion parameter under each time slice, and obtain the matched palm overall motion parameter and the fingertip local motion parameter under all time slices after spatial calibration. The matched palm overall motion parameter and the fingertip local motion parameter under all time slices after spatial calibration are input into the pre-trained action correlation model one by one to calculate action correlation strength, and the corresponding action correlation strength under all time slices is obtained.

[0010] According to the above description, when the palm overall motion parameter and the fingertip local motion parameter are time-sliced, the stage correlation mapping rule constructed by the palm overall motion and the fingertip local motion is introduced to avoid stage mismatch caused by time slicing according to the current collection timestamp, and the rationality of the matched palm overall motion parameter and the fingertip local motion parameter under each time slice is ensured. The hand three-dimensional coordinate system is established to realize spatial calibration of the matched palm overall motion parameter and the fingertip local motion parameter under each time slice, which can effectively eliminate the influence of the size difference of different students' hands on the motion parameter, and improve the accuracy and objectivity of the calculated action correlation strength.

[0011] Optionally, the inputting of the matched palm overall motion parameter and the fingertip local motion parameter under all time slices after spatial calibration into the pre-trained action correlation model to calculate action correlation strength and obtain the corresponding action correlation strength under all time slices comprises: The matched palm overall motion parameter and the fingertip local motion parameter under all time slices after spatial calibration are input into the pre-trained action correlation model one by one, and constraint deviations of the matched palm overall motion parameter and the fingertip local motion parameter under each time slice under each type of constraint condition are calculated by four types of preset constraint conditions built in the action correlation model, so as to obtain four types of constraint deviations under all time slices, the four types of preset constraint conditions include displacement constraint condition, velocity constraint condition, acceleration constraint condition and posture constraint condition, and the four types of constraint deviations include displacement constraint deviation, velocity constraint deviation, acceleration constraint deviation and posture constraint deviation. The displacement constraint condition is: ; Wherein, represents the displacement constraint deviation, represents a tangent direction included angle of a palm overall motion trajectory in the matched palm overall motion parameter under time slice i, denotes a tangent direction angle between a local motion trajectory of a fingertip in the paired fingertip local motion parameters under the time slice i, denotes a first threshold value; The speed constraint condition is: ; ; wherein, denotes a global motion speed of a palm in the paired palm global motion parameters under the time slice i, denotes a local motion speed of a fingertip in the paired fingertip local motion parameters under the time slice i, denotes a speed constraint deviation, denotes a first coefficient, denotes a second coefficient; The acceleration constraint condition is: ; wherein, denotes an acceleration constraint deviation, denotes a global acceleration peak value of a palm in the paired palm global motion parameters under the time slice i, denotes a local acceleration peak value of a fingertip in the paired fingertip local motion parameters under the time slice i, denotes a second threshold value; The posture constraint condition is: ; ; wherein, denotes a global posture angular rate of change of a palm in the paired palm global motion parameters under the time slice i, denotes a local posture angular rate of change of a fingertip in the paired fingertip local motion parameters under the time slice i, denotes a third threshold value, denotes a fourth threshold value, denotes a posture constraint deviation; The action correlation strength under each time slice is calculated by substituting the corresponding four types of constraint deviations under all time slices into the weighted formula one by one, wherein the weighted formula is: ; wherein, denotes the action correlation strength under the time slice i, denotes a first weight, denotes a second weight, denotes a third weight, denotes a fourth weight.

[0012] According to the above description, the constraint deviations of the paired palm overall motion parameters and fingertip local motion parameters under each constraint condition are calculated by the four types of preset constraint conditions in the action correlation model, i.e., the action correlation strength of the palm overall motion and the fingertip local motion is calculated from four aspects of displacement constraint condition, velocity constraint condition, acceleration constraint condition and posture constraint condition, which solves the problem that a single dimension cannot comprehensively reflect the action coordination, and quantifies various constraint deviations through explicit mathematical formulas, replaces subjective judgment, and makes the calculation of the action correlation strength more objective and scientific.

[0013] Optionally, the fusion processing of the action-coordinated palm overall motion parameters and the action-coordinated fingertip local motion parameters and the pressure parameters to obtain the dual-mode fusion feature includes: calculating the time difference of the action-coordinated palm overall motion parameters and the action-coordinated fingertip local motion parameters at a preset key node under the same time slice, obtaining a time difference corresponding to each time slice, judging whether the time difference corresponding to each time slice is lower than a first threshold value, if yes, taking the time slice with the time difference lower than the first threshold value as a candidate time slice; calculating the ratio of the overall pressure change rate in the palm overall pressure parameter to the local pressure change rate in the fingertip local pressure parameter under each candidate time slice, obtaining a ratio corresponding to each candidate time slice, judging whether the ratio corresponding to each candidate time slice is located in a first preset interval, if not, filtering the candidate time slice with the ratio not located in the first preset interval to obtain an effective time slice; fusion processing of the action-coordinated palm overall motion parameters and the action-coordinated fingertip local motion parameters under the effective time slice and the pressure parameters to obtain a dual-mode fusion feature.

[0014] According to the above description, the candidate time slice is first screened through the time difference of the preset key node, and then the effective time slice is screened through the ratio of the overall pressure change rate to the local pressure change rate, which can accurately exclude the motion parameters with insufficient action coordination, not only reduces the data amount and improves the efficiency of fusion processing, but also focuses on the motion parameters under the effective time slice when performing fusion processing, and improves the reliability of the dual-mode fusion feature.

[0015] Optionally, the fusion processing of the action-coordinated palm overall motion parameters and the action-coordinated fingertip local motion parameters under the effective time slice and the pressure parameters to obtain a dual-mode fusion feature includes: Based on the overall palm motion parameters and the local fingertip motion parameters under the effective time slice, a preliminary technique type is identified to obtain a preliminary technique type. Based on the preliminary technique type, a matching standard pressure distribution entropy is called from a pre-built standard technique library. The overall palm pressure distribution entropy in the overall palm pressure parameters and the local fingertip pressure distribution entropy in the local fingertip pressure parameters are integrated to obtain the current pressure distribution entropy. The current pressure distribution entropy, the standard pressure distribution entropy, and the effective time slice are input into the first synchronization rate formula to calculate the synchronization rate of hand and fingertip movements. The first synchronization rate formula is as follows: ; ; in, This indicates the synchronization rate between the movements of the palm and fingertips. This represents the pressure distribution entropy correction coefficient. This represents the current pressure distribution entropy. Represents the standard pressure distribution entropy. Indicates the number of valid time slices. Indicates the total number of time slices; The average pressure value of the entire palm from the overall palm pressure parameters and the average pressure value of the fingertip from the local fingertip pressure parameters are integrated to obtain the current average pressure value sequence. Based on the preliminary technique type, a matching standard speed sequence is retrieved from a pre-built standard technique library. The standard speed sequence and the current average pressure value sequence are input into a second synchronization rate formula for calculation to obtain the force-velocity synchronization rate of pressure and speed. The second synchronization rate formula is: ; in, The force-velocity synchronization rate represents the relationship between pressure and velocity. This represents the Pearson correlation coefficient between the current average pressure series and the standard velocity series. This represents the current average pressure value sequence. Represents a standard velocity sequence. This represents the dynamic time-warped distance between the current average pressure value sequence and the standard velocity sequence. This represents the largest dynamic time warp distance in the standard technique library.

[0016] As described above, preliminary technique type identification is performed based on the overall hand motion parameters and the local fingertip motion parameters within the effective time slice. This allows for the retrieval of matching standard pressure distribution entropy and standard velocity sequences based on the preliminary technique type, ensuring the rationality of the baseline for calculating the synchronization rate of hand and fingertip movements, as well as the force-velocity synchronization rate of pressure and velocity. By integrating the overall hand pressure distribution entropy and the local fingertip pressure distribution entropy for calculating the synchronization rate of hand and fingertip movements, the synchronization rate reflects not only temporal coordination but also the rationality of pressure distribution, improving the accuracy of the synchronization rate calculation. In calculating the force-velocity synchronization rate of pressure and velocity, the Pearson correlation coefficient, representing linear correlation, and the dynamic time warping distance, representing temporal similarity, are used to comprehensively reflect the coordination quality of pressure and velocity, improving the scientific rigor of the force-velocity synchronization rate calculation.

[0017] Optionally, the step of inputting the dual-mode fusion features, the pressure parameters, the overall hand motion parameters coordinated by the movement, and the local fingertip motion parameters coordinated by the movement as the current massage parameters into the pre-trained massage technique recognition model to perform massage technique recognition, thereby obtaining the current massage technique type, and calling the matching standard massage parameters from the pre-built standard technique library according to the current massage technique type, and calculating the difference between the standard massage parameters and the current massage parameters includes: The current massage parameters are normalized to the [0,1] interval using Min-Max normalization to achieve standardization of the current massage parameters and obtain the standardized current massage parameters. The random forest algorithm is used to select N core parameters from the standardized current massage parameters, and the principal component analysis algorithm is used to reduce the dimensionality of the N core parameters to obtain the N core parameters after dimensionality reduction. The N core parameters after dimensionality reduction are input into the pre-trained massage technique recognition model to identify the massage technique and obtain the current massage technique type. The pre-trained massage technique recognition model is built and trained based on an LSTM network. Based on the current massage technique type, N standard massage parameters that match the N core parameters after dimensionality reduction are called from the pre-built standard technique library, and the difference between the N standard massage parameters and the N core parameters after dimensionality reduction is calculated.

[0018] As described above, Min-Max normalization is used to standardize the current massage parameters. Then, the core parameters are selected through the random forest algorithm and dimensionality is reduced by principal component analysis. This reduces the interference of redundant parameters on the massage technique recognition model and lowers the computational complexity of the model. The massage technique recognition model based on LSTM network can effectively capture the temporal dependency of the core parameters during massage and improve the accuracy of massage technique recognition.

[0019] Optionally, the difference between the calculated N standard massage parameters and the N core parameters after dimensionality reduction includes: Obtain the parameter type for each core parameter. When the parameter type is numerical, use the difference formula to calculate the difference between the core parameter and the corresponding standard massage parameter. When the parameter type is directional, the difference between the core parameter and the corresponding standard massage parameter is calculated using the vector angle cosine theorem. When the parameter type is time series, determine whether the time series is a periodic uniform time series. If yes, use the coefficient of variation method to calculate the difference between the core parameter and the corresponding standard massage parameter. If no, use the waveform similarity algorithm to calculate the difference between the core parameter and the corresponding standard massage parameter. When the parameter type is stable, the difference between the core parameter and the corresponding standard massage parameter is calculated using the standard deviation formula.

[0020] As described above, the difference is calculated using a type-adaptive method for different types of core parameters to ensure the scientific validity and accuracy of the calculated difference. Furthermore, the categorized calculation facilitates the subsequent identification of core parameters with differences, providing support for personalized guidance.

[0021] Secondly, the present invention provides a massage teaching system based on multimodal data collaboration, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the massage teaching method based on multimodal data collaboration described in the first aspect.

[0022] The technical effects of the massage teaching system based on multimodal data collaboration provided in the second aspect are the same as those of the massage teaching method based on multimodal data collaboration provided in the first aspect. Attached Figure Description

[0023] Figure 1 A flowchart illustrating a massage teaching method based on multimodal data collaboration provided in this embodiment; Figure 2This is a schematic diagram of the overall process of a massage teaching method based on multimodal data collaboration provided in this embodiment; Figure 3 This is a schematic diagram of the structure of a massage teaching system based on multimodal data collaboration provided in this embodiment.

[0024] Explanation of reference numerals in the attached figures 1. A massage teaching system based on multimodal data collaboration; 2. Processor; 3. Memory. Detailed Implementation

[0025] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0026] Example 1 Please refer to Figures 1 to 2 This invention provides a massage teaching method based on multimodal data collaboration, comprising the following steps: S1. Real-time acquisition of pressure distribution data and kinematic data during the student's massage process; preprocessing of the pressure distribution data and kinematic data to obtain preprocessed pressure distribution data and preprocessed kinematic data; extraction of pressure features from the preprocessed pressure distribution data to obtain pressure parameters; and extraction of motion features from the preprocessed kinematic data to obtain motion parameters. The kinematic data includes overall palm motion data and fingertip local motion data; the pressure parameters include overall palm pressure parameters and fingertip local pressure parameters; and the motion parameters include overall palm motion parameters and fingertip local motion parameters. In this embodiment, the trainee wears a smart data glove during the massage. This smart data glove integrates a flexible thin-film pressure sensor array and multiple inertial measurement units (IMUs). The flexible thin-film pressure sensor array is arranged in a biomimetic arc shape: "3 points at the fingertips + 6×8 array on the thenar eminence + 6×8 array on the hypothenar eminence," conforming to the physiological curve of the palm. The pressure detection range of a single sensor is 0-50N, with an accuracy of ±0.1N, a sampling rate ≥100Hz, and a response time ≤8ms. The IMUs adopt a "center of the palm + fingertips" layout, with one 6-axis IMU located at the center of the back of the hand and at the tips of the thumb and index finger. The sampling frequency of the central IMU is ≥100Hz, with an acceleration measurement error ≤±0.05g and an angular velocity measurement error ≤±0.1° / s. The sampling frequency of the peripheral IMUs is ≥200Hz.

[0027] like Figure 2 As shown, the smart data glove can actually collect and transmit pressure distribution data and kinematic data of the trainee during the massage process. The kinematic data includes overall palm movement data and fingertip local movement data. The collected pressure distribution data and kinematic data are preprocessed to obtain preprocessed pressure distribution data and preprocessed kinematic data, respectively. Preprocessing includes, but is not limited to, noise reduction and filtering. Pressure features are extracted from the preprocessed pressure distribution data to obtain pressure parameters, and motion features are extracted from the preprocessed kinematic data to obtain motion parameters. The pressure parameters include overall palm pressure parameters and fingertip local pressure parameters, and the motion parameters include overall palm movement parameters and fingertip local movement parameters.

[0028] At this point, the preprocessing in step S1 includes noise reduction processing. The pressure distribution data and the kinematic data are preprocessed separately to obtain preprocessed pressure distribution data and preprocessed kinematic data. Pressure features are extracted from the preprocessed pressure distribution data to obtain pressure parameters. Simultaneously, motion features are extracted from the preprocessed kinematic data to obtain motion parameters, including: S11. The pressure distribution data is denoised by median filtering and Gaussian smoothing respectively to obtain denoised pressure distribution data. Based on the preset hand contact area division rule, the denoised pressure distribution data is divided into palm area pressure distribution data and fingertip local pressure distribution data. S12. The pressure distribution data of the palm region and the local pressure distribution data of the fingertips are converted into two-dimensional pressure heat maps by bilinear interpolation, respectively, to obtain a two-dimensional pressure heat map of the whole palm and a two-dimensional pressure heat map of the local fingertips. Then, the voxel reconstruction technology is used to reconstruct the two-dimensional pressure heat map of the whole palm and the two-dimensional pressure distribution map of the local fingertips into three-dimensional pressure heat maps, respectively, to obtain a three-dimensional pressure heat map of the whole palm and a three-dimensional pressure heat map of the local fingertips. Pressure features are extracted from the three-dimensional pressure heat map of the whole palm and the three-dimensional pressure heat map of the local fingertips to obtain the overall pressure parameters of the palm and the local pressure parameters of the fingertips. S13. The overall hand motion data is denoised using Kalman filtering to obtain denoised overall hand motion data. Simultaneously, the local fingertip motion data is denoised using particle filtering to obtain denoised local fingertip motion data. Motion features are extracted from the denoised overall hand motion data and the denoised local fingertip motion data to obtain the corresponding overall hand motion parameters and local fingertip motion parameters.

[0029] In this embodiment, as Figure 2As shown, a hybrid algorithm is used to perform secondary denoising on the pressure distribution data. Median filtering and Gaussian smoothing are applied to denoise the pressure distribution data, resulting in denoised pressure distribution data. Based on a preset hand contact area segmentation rule, the denoised pressure distribution data is divided into palm region pressure distribution data and fingertip local pressure distribution data. Specifically, the preset hand contact area segmentation rule is to divide the data into regions of interest (RIOs) based on the palm and fingertip contact areas. Bilinear interpolation is used to convert the palm region pressure distribution data and fingertip local pressure distribution data into two-dimensional pressure heatmaps. Voxel reconstruction technology is then used to reconstruct the overall palm two-dimensional pressure heatmap and the fingertip local two-dimensional pressure heatmap into overall palm three-dimensional pressure heatmap and fingertip local three-dimensional pressure heatmap, respectively. Pressure features are extracted from both the overall palm three-dimensional pressure heatmap and the fingertip local three-dimensional pressure heatmap to obtain overall palm pressure parameters and fingertip local pressure parameters. The overall palm pressure parameters include, but are not limited to: peak value of overall palm pressure, average overall palm pressure, entropy of overall palm pressure distribution, rise time of overall palm pressure, fall time of overall palm pressure, area of ​​overall palm pressure, and gradient of overall palm pressure. The local fingertip pressure parameters include, but are not limited to: peak value of local fingertip pressure, average local fingertip pressure, entropy of local fingertip pressure distribution, rise time of local fingertip pressure, fall time of local fingertip pressure, area of ​​local fingertip pressure, and gradient of local fingertip pressure.

[0030] Different algorithms were used to denoise different kinematic data. Kalman filtering was used to denoise the overall palm motion data, while particle filtering was used to denoise the fingertip local motion data, resulting in denoised overall palm motion data and fingertip local motion data. Motion feature extraction was then performed on both denoised data to obtain the corresponding overall palm motion parameters and fingertip local motion parameters. The overall palm motion parameters include, but are not limited to: the trajectory of the overall palm motion, the curvature of the trajectory, the peak velocity, the peak angular velocity, the amplitude of the attitude angle change, the motion period, the peak acceleration, the trajectory length, and the motion uniformity. The fingertip local motion parameters include, but are not limited to: the trajectory of the fingertip local motion, the curvature of the trajectory, the peak velocity, the peak angular velocity, the amplitude of the attitude angle change, the motion period, the peak acceleration, the trajectory length, and the motion uniformity. The trajectory, velocity, and acceleration parameters were calculated using quaternion integration.

[0031] S2. Obtain the current acquisition timestamp, and perform time slices on the overall hand motion parameters and the local fingertip motion parameters according to the current acquisition timestamp to obtain the overall hand motion parameters and local fingertip motion parameters under all time slices. Input the overall hand motion parameters and local fingertip motion parameters under each time slice into the pre-trained motion association model to calculate the motion association strength, and obtain the corresponding motion association strength under all time slices. Filter out the coordinated overall hand motion parameters and coordinated local fingertip motion parameters according to the corresponding motion association strength under all time slices. At this point, in step S2, the overall hand motion parameters and the local fingertip motion parameters are sliced ​​according to the current acquisition timestamp to obtain the overall hand motion parameters and local fingertip motion parameters for all time slices. The overall hand motion parameters and local fingertip motion parameters for each time slice are then input into the pre-trained motion association model to calculate the motion association strength, resulting in the motion association strength for all time slices, including: S21. Using the motion cycle of the overall palm motion corresponding to the preprocessed overall palm motion data as the time reference, the overall palm motion is divided into overall stages to obtain the overall stage time sequence. Based on the fingertip local motion parameters, the fingertip local motion corresponding to the preprocessed fingertip local motion data is divided into local stages to obtain the local stage time sequence. The local stage time sequence is associated and mapped with the overall stage time sequence to construct the stage association mapping rule between the overall palm motion and the fingertip local motion. In this embodiment, as Figure 2 As shown, using the motion cycle of the preprocessed overall hand motion data as the time base, the overall hand motion is divided into stages, such as the pressure application stage, the maintenance stage, and the decompression stage, resulting in an overall stage time sequence. Based on the fingertip local motion parameters, the preprocessed fingertip local motion data is divided into local stages, such as the inflection points of the motion trajectory and the abrupt changes in velocity, resulting in the contact sub-stage, the maintenance sub-stage, and the ion withdrawal stage, resulting in a local stage time sequence. The local stage time sequences are then correlated and mapped with the overall stage time sequences, such as the pressure application stage corresponding to the contact sub-stage, the maintenance stage corresponding to the maintenance sub-stage, and the decompression stage corresponding to the ion withdrawal stage, to construct a stage correlation mapping rule between the overall hand motion and the fingertip local motion.

[0032] S22. According to the stage association mapping rule and the current collection timestamp, the overall palm motion parameters and the local fingertip motion parameters are time-sliced ​​according to a preset time slice to obtain the overall palm motion parameters and local fingertip motion parameters under all time slices. S23. Establish a three-dimensional coordinate system for the hand with the center coordinate of the back of the hand in the overall hand motion parameters paired in each time slice as the origin coordinate. Based on the three-dimensional coordinate system of the hand, convert the fingertip coordinates in the corresponding fingertip local motion parameters into relative coordinates to achieve spatial calibration of the overall hand motion parameters and fingertip local motion parameters paired in each time slice, and obtain the spatially calibrated overall hand motion parameters and fingertip local motion parameters paired in all time slices. S24. Input the overall hand motion parameters and fingertip local motion parameters of all time slices after spatial calibration into the pre-trained motion association model to calculate the motion association strength, and obtain the corresponding motion association strength for all time slices.

[0033] In this embodiment, as Figure 2 As shown, based on the stage association mapping rules constructed in step S21 and the current acquisition timestamp, the overall palm motion parameters and fingertip local motion parameters are time-sliced ​​according to preset time slices to obtain the overall palm motion parameters and fingertip local motion parameters under all time slices. That is, time synchronization of the overall palm motion parameters and fingertip local motion parameters is achieved. A three-dimensional coordinate system for the hand is established with the center coordinate of the back of the hand in the paired overall palm motion parameters under each time slice as the origin coordinate. Based on the three-dimensional coordinate system of the hand, the fingertip coordinates in the corresponding fingertip local motion parameters are converted into relative coordinates to achieve spatial calibration of the paired overall palm motion parameters and fingertip local motion parameters under each time slice, obtaining the spatially calibrated overall palm motion parameters and fingertip local motion parameters under all time slices. That is, spatial calibration of the overall palm motion parameters and fingertip local motion parameters is achieved. The spatially calibrated overall palm motion parameters and fingertip local motion parameters under all time slices are then input one by one into the pre-trained action association model to calculate the action association strength, obtaining the corresponding action association strength under all time slices.

[0034] At this point, step S24 includes: S241. Input the overall hand motion parameters and fingertip local motion parameters paired under all time slices after spatial calibration into the pre-trained motion association model one by one. Calculate the constraint deviation of the overall hand motion parameters and fingertip local motion parameters paired under each time slice under each type of constraint condition using the four types of preset constraints built into the motion association model, so as to obtain the four types of constraint deviations corresponding to all time slices. The four types of preset constraints include displacement constraint conditions, velocity constraint conditions, acceleration constraint conditions, and posture constraint conditions. The four types of constraint deviations include displacement constraint deviation, velocity constraint deviation, acceleration constraint deviation, and posture constraint deviation. The displacement constraint condition is as follows: ; in, Indicates displacement constraint deviation. This represents the angle between the tangent directions of the overall hand motion trajectory in the paired overall hand motion parameters at time slice i. This represents the angle between the tangent directions of the fingertip local motion trajectory in the paired fingertip local motion parameters under time slice i. Indicates the first threshold; The speed constraint condition is: ; ; in, This represents the overall hand velocity in the overall hand motion parameters paired at time slice i. This represents the local velocity of the fingertip in the paired local motion parameters for time slice i. Indicates speed constraint deviation. Indicates the first coefficient. Indicates the second coefficient; The acceleration constraint condition is as follows: ; in, Indicates acceleration constraint deviation. This represents the peak value of the overall hand acceleration in the overall hand motion parameters paired at time slice i. This represents the peak value of the local fingertip acceleration in the paired local fingertip motion parameters at time slice i. Indicates the second threshold; The attitude constraint is as follows: ; ; in, This represents the rate of change of the overall hand posture angle in the overall hand motion parameters paired at time slice i. This represents the rate of change of the local fingertip posture angle among the paired local fingertip motion parameters at time slice i. This represents the third threshold. This represents the fourth threshold. Indicates attitude constraint deviation; S242. Substitute the four types of constraint deviations corresponding to all time slices into the weighting formula to calculate the motion association strength, obtaining the motion association strength corresponding to all time slices. The weighting formula is as follows: ; in, Indicates the action correlation strength under time slice i. Indicates the first weight. Indicates the second weight. Indicates the third weight. This indicates the fourth weight.

[0035] In this embodiment, as Figure 2 As shown, the overall hand motion parameters and fingertip local motion parameters paired under all time slices after spatial calibration are input one by one into the pre-trained motion association model. The motion association model has four types of preset constraints: displacement constraints, velocity constraints, acceleration constraints, and posture constraints. The constraint deviations of the overall hand motion parameters and fingertip local motion parameters paired under each type of constraint are calculated for each time slice to obtain the four types of constraint deviations for all time slices: displacement constraint deviation, velocity constraint deviation, acceleration constraint deviation, and posture constraint deviation. The four types of constraint deviations for all time slices are then substituted into the weighted formula to calculate the motion association strength for all time slices. The sum of the first, second, third, and fourth weights in the weighted formula is 1. Each weight is obtained by training the motion association model using standard operation data from multiple sets of expert massage procedures. Furthermore, when subsequently filtering out the overall hand motion parameters and coordinated fingertip local motion parameters based on the motion correlation strength across all time slices, the process involves comparing the motion correlation strength for each time slice with a correlation threshold. Time slices with motion correlation strengths below the threshold are filtered out, meaning that time slices with motion correlation strengths above the threshold are considered to have coordinated overall hand motion parameters and coordinated fingertip local motion parameters. The correlation threshold is set at 0.8, but can be adjusted based on specific circumstances.

[0036] S3. The overall motion parameters of the hand and the local motion parameters of the fingertips are fused with the pressure parameters to obtain dual-mode fusion features. The dual-mode fusion features include the motion synchronization rate of the hand and fingertips and the force-velocity synchronization rate of pressure and speed. At this point, the process of fusing the overall hand motion parameters and the local fingertip motion parameters with the pressure parameters in step S3 to obtain the dual-mode fusion features includes: S31. Calculate the time difference between the overall motion parameters of the palm and the local motion parameters of the fingertips in the same time slice at the preset key nodes, obtain the time difference corresponding to each time slice, and determine whether the time difference corresponding to each time slice is lower than the first threshold. If so, the time slice corresponding to the time difference lower than the first threshold is taken as the candidate time slice. S32. Calculate the ratio of the overall pressure change rate in the overall palm pressure parameter to the local pressure change rate in the local fingertip pressure parameter under each candidate time slice, obtain the ratio corresponding to each candidate time slice, determine whether the ratio corresponding to each candidate time slice is within the first preset interval, if not, filter out the candidate time slices whose ratio is not within the first preset interval to obtain the effective time slices. S33. The overall hand motion parameters and the local fingertip motion parameters under the effective time slice are fused with the pressure parameters to obtain dual-mode fusion features.

[0037] In this embodiment, as Figure 2 As shown, the time difference between the overall hand motion parameters and the local fingertip motion parameters in the same time slice is calculated at preset key nodes. These preset key nodes include: the pressure initiation time, pressure peak time, and pressure release initiation time of the overall hand motion; and the contact time, pressure peak time, and withdrawal time of the local fingertip motion. This yields the time difference for each time slice. Time slices with time differences lower than a first threshold are designated as candidate time slices, thus further synchronizing the time. The ratio of the overall pressure change rate in the overall hand motion parameters to the local pressure change rate in the local fingertip pressure parameters is calculated for each candidate time slice. Candidate time slices with ratios not falling within the first preset interval are filtered out, achieving intensity synchronization. The first preset interval is [0.9, 1.1], and this can be adjusted based on actual conditions. The overall hand motion parameters and local fingertip motion parameters in all effective time slices are fused with the pressure parameters to obtain dual-mode fusion features.

[0038] At this point, step S33 includes: S331. Based on the overall palm motion parameters and the local fingertip motion parameters of the coordinated movement under the effective time slice, a preliminary manual technique type is identified to obtain a preliminary manual technique type. Based on the preliminary manual technique type, a matching standard pressure distribution entropy is called from the pre-built standard manual technique library, and the overall palm pressure distribution entropy in the overall palm pressure parameters and the local fingertip pressure distribution entropy in the local fingertip pressure parameters are integrated to obtain the current pressure distribution entropy. S332. Input the current pressure distribution entropy, the standard pressure distribution entropy, and the effective time slice into the first synchronization rate formula for calculation to obtain the synchronization rate of the hand and fingertips. The first synchronization rate formula is: ; ; in, This indicates the synchronization rate between the movements of the palm and fingertips. This represents the pressure distribution entropy correction coefficient. This represents the current pressure distribution entropy. Represents the standard pressure distribution entropy. Indicates the number of valid time slices. Indicates the total number of time slices; S333. Integrate the overall palm pressure parameter with the fingertip pressure parameter to obtain the current average pressure value sequence. Based on the preliminary technique type, retrieve the matching standard speed sequence from the pre-built standard technique library. Input the standard speed sequence and the current average pressure value sequence into the second synchronization rate formula for calculation to obtain the force-speed synchronization rate of pressure and speed. The second synchronization rate formula is: ; in, The force-velocity synchronization rate represents the relationship between pressure and velocity. This represents the Pearson correlation coefficient between the current average pressure series and the standard velocity series. This represents the current average pressure value sequence. Represents a standard velocity sequence. This represents the dynamic time-warped distance between the current average pressure value sequence and the standard velocity sequence. This represents the largest dynamic time warp distance in the standard technique library.

[0039] In this embodiment, preliminary technique type identification is performed based on the overall palm motion parameters and the local fingertip motion parameters under the effective time slice. This is done using a pre-trained lightweight motion classification model. Based on the preliminary technique type, a matching standard pressure distribution entropy is retrieved from a pre-built standard technique library. This library stores standard massage parameters for different technique types from experts. The overall palm pressure distribution entropy and the local fingertip pressure distribution entropy are integrated to obtain the current pressure distribution entropy. The current pressure distribution entropy, the standard pressure distribution entropy, and the effective time slice are input into a first synchronization rate formula to calculate the synchronization rate between the palm and fingertips. The overall average palm pressure value and the local average fingertip pressure value are integrated to obtain the current average pressure value sequence. Similarly, based on the preliminary technique type, a matching standard speed sequence is retrieved from the pre-built standard technique library. The standard speed sequence and the current average pressure value are input into a second synchronization rate formula to calculate the pressure and appropriate force-speed synchronization rate.

[0040] S4. The dual-mode fusion features, the pressure parameters, the overall hand motion parameters of the coordinated movement, and the local fingertip motion parameters of the coordinated movement are used as the current massage parameters to input into the pre-trained massage technique recognition model to perform massage technique recognition, thereby obtaining the current massage technique type. Based on the current massage technique type, the matching standard massage parameters are called from the pre-built standard technique library. The difference between the standard massage parameters and the current massage parameters is calculated. Based on the difference, the current massage score is generated, and personalized massage teaching guidance suggestions are generated based on the current massage score.

[0041] In this embodiment, as Figure 2 As shown, the dual-mode fusion features, pressure parameters, overall hand motion parameters, and fingertip local motion parameters are used as inputs to a pre-trained massage technique recognition model to identify massage techniques. This further refines the massage technique identification to determine the current massage technique type. Based on the current massage technique type, matching standard massage parameters are retrieved from a pre-built standard technique library. The difference between the standard massage parameters and the current massage parameters is calculated, and a current massage score is generated based on this difference to further generate personalized massage teaching guidance suggestions.

[0042] At this point, step S4 involves inputting the dual-mode fusion features, the pressure parameters, the overall hand motion parameters coordinated by the movement, and the local fingertip motion parameters coordinated by the movement as the current massage parameters into the pre-trained massage technique recognition model to identify the massage technique, thereby obtaining the current massage technique type. Based on the current massage technique type, matching standard massage parameters are retrieved from the pre-built standard technique library, and the difference between the standard massage parameters and the current massage parameters is calculated, including: S41. The current massage parameters are normalized to the [0,1] interval using Min-Max normalization to achieve the standardization of the current massage parameters and obtain the standardized current massage parameters. S42. Use the random forest algorithm to select N core parameters from the standardized current massage parameters, and use the principal component analysis algorithm to reduce the dimensionality of the N core parameters to obtain the N core parameters after dimensionality reduction. S43. Input the N core parameters after dimensionality reduction into the pre-trained massage technique recognition model to recognize the massage technique and obtain the current massage technique type. The pre-trained massage technique recognition model is built and trained based on LSTM network. S44. Based on the current massage technique type, retrieve N standard massage parameters from the pre-built standard technique library that match the N core parameters after dimensionality reduction, and calculate the degree of difference between the N standard massage parameters and the N core parameters after dimensionality reduction.

[0043] In this embodiment, the current massage parameters are standardized, and a random forest algorithm is used to select N core parameters from the standardized parameters. N is 10, which can be adjusted according to the actual situation. Principal component analysis is then used to reduce the dimensionality of the N core parameters, resulting in N dimensionality-reduced core parameters. These N dimensionality-reduced core parameters are then input into a massage technique recognition model built and trained based on an LSTM network to identify the current massage technique type. Therefore, based on the current massage technique type, N standard massage parameters matching the N dimensionality-reduced core parameters are retrieved from a pre-built standard technique library, and the difference between each standard massage parameter and its corresponding dimensionality-reduced core parameter is calculated.

[0044] At this point, the calculation of the difference between the N standard massage parameters and the N core parameters after dimensionality reduction in step S44 includes: S441. Obtain the parameter type of each core parameter. When the parameter type is numerical, use the difference formula to calculate the difference between the core parameter and the corresponding standard massage parameter. S442. When the parameter type is directional, the difference between the core parameter and the corresponding standard massage parameter is calculated using the vector angle cosine theorem. S443. When the parameter type is time series, determine whether the time series is a periodic uniform time series. If so, use the coefficient of variation method to calculate the difference between the core parameter and the corresponding standard massage parameter. If not, use the waveform similarity algorithm to calculate the difference between the core parameter and the corresponding standard massage parameter. S444. When the parameter type is stable, the difference between the core parameter and the corresponding standard massage parameter is calculated using the standard deviation formula.

[0045] In this embodiment, when calculating the difference between the standard massage parameters and the N core parameters after dimensionality reduction, different difference calculation methods are selected according to the parameter type of the core parameters. When the parameter type of the core parameters is numerical, the difference formula is used to calculate the difference between the core parameters and the corresponding standard massage parameters. This difference formula can be either an absolute value difference formula or a relative value difference formula. The absolute value difference formula is as follows: Difference = |Core Parameters - Standard Massage Parameters|; The formula for the relative value difference is: Difference = |Core Parameters - Standard Massage Parameters| / Standard Massage Parameters; When the parameter type is directional, the difference between the core parameter and the corresponding standard massage parameter is calculated using the vector angle cosine theorem. The vector angle cosine theorem in this case is: ; in, This represents the angle between the force direction vector of the core parameter and the force direction vector of the standard massage parameter; When the parameter type is time-series, if the time-series type specifically refers to periodic uniformity, then the coefficient of variation method is used to calculate the difference between the core parameters and the corresponding standard massage parameters. The coefficient of variation method is as follows: ; If the temporal sequence is not periodically uniform, a waveform similarity algorithm is used to calculate the difference between the core parameters and the corresponding standard massage parameters. The waveform similarity algorithm is as follows: ; If the parameter type is stable, the difference between the core parameter and the corresponding standard massage parameter is calculated using the standard deviation formula, where the standard deviation formula is: ; After calculating N differences, these are substituted into a weighted summation formula to generate the current massage score. Based on this score, personalized massage teaching guidance suggestions are then generated. For example, if the current massage score is below the first scoring threshold, the specific core parameters indicating differences are identified, and corresponding massage teaching guidance suggestions are retrieved from the teaching guidance suggestion library.

[0046] Example 2 Please refer to Figure 3 The present invention provides a massage teaching system 1 based on multimodal data collaboration, including a memory 3, a processor 2, and a computer program stored on the memory 3 and run on the processor 2. When the processor 2 executes the computer program, it implements the steps in Embodiment 1.

[0047] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0048] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0049] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0050] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0051] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0052] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0053] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A massage teaching method based on multi-modal data collaboration, characterized in that, The method comprises the following steps: real-time acquisition of pressure distribution data and kinematics data during the massage of a student, pre-processing of the pressure distribution data and the kinematics data respectively to obtain pre-processed pressure distribution data and pre-processed kinematics data, pressure feature extraction on the pre-processed pressure distribution data to obtain pressure parameters, kinematics feature extraction on the pre-processed kinematics data to obtain kinematics parameters, the kinematics data comprising palm overall kinematics data and fingertip local kinematics data, the pressure parameters comprising palm overall pressure parameters and fingertip local pressure parameters, and the kinematics parameters comprising palm overall kinematics parameters and fingertip local kinematics parameters; acquisition of a current acquisition timestamp, time slicing of the palm overall kinematics parameters and the fingertip local kinematics parameters according to the current acquisition timestamp to obtain palm overall kinematics parameters and fingertip local kinematics parameters under all time slices, inputting the palm overall kinematics parameters and the fingertip local kinematics parameters under each time slice into a pre-trained action correlation model one by one to calculate action correlation strength to obtain corresponding action correlation strength under all time slices, and screening out action-coordinated palm overall kinematics parameters and action-coordinated fingertip local kinematics parameters according to the corresponding action correlation strength under all time slices; fusion processing of the action-coordinated palm overall kinematics parameters and the action-coordinated fingertip local kinematics parameters with the pressure parameters to obtain dual-mode fusion features, the dual-mode fusion features comprising action synchronization rate of the palm and the fingertips and force-velocity synchronization rate of the pressure and the speed; inputting the dual-mode fusion features, the pressure parameters, the action-coordinated palm overall kinematics parameters and the action-coordinated fingertip local kinematics parameters together as current massage parameters into a pre-trained massage technique recognition model to perform massage technique recognition to obtain a current massage technique type, calling matched standard massage parameters from a pre-constructed standard technique library according to the current massage technique type, calculating a difference degree between the standard massage parameters and the current massage parameters, generating a current massage score according to the difference degree, and generating an individualized massage teaching guidance suggestion according to the current massage score.

2. The massage teaching method based on multi-modal data collaboration according to claim 1, characterized in that, The pre-processing comprises noise reduction processing, and the pre-processing of the pressure distribution data and the kinematics data respectively to obtain pre-processed pressure distribution data and pre-processed kinematics data, and the pressure feature extraction on the pre-processed pressure distribution data to obtain pressure parameters, and the kinematics feature extraction on the pre-processed kinematics data to obtain kinematics parameters comprises: noise reduction processing of the pressure distribution data by a median filtering method and a Gaussian smoothing method respectively to obtain noise-reduced pressure distribution data, and division of the noise-reduced pressure distribution data into palm region pressure distribution data and fingertip local pressure distribution data based on a pre-set hand contact region division rule; The palm region pressure distribution data and the fingertip local pressure distribution data are converted into two-dimensional pressure thermal maps by a bilinear interpolation method, and a palm overall two-dimensional pressure thermal map and a fingertip local two-dimensional pressure distribution map are obtained. The palm overall motion data is denoised by a Kalman filtering method to obtain denoised palm overall motion data, and the fingertip local motion data is denoised by a particle filtering method to obtain denoised fingertip local motion data.

3. The massage teaching method based on multi-modal data collaboration according to claim 1, characterized in that, The palm overall motion parameters and the fingertip local motion parameters are time-sliced according to the current collection timestamp to obtain palm overall motion parameters and fingertip local motion parameters under all time slices. The palm overall motion is divided into overall stages based on the motion period of the palm overall motion corresponding to the preprocessed palm overall motion data, and an overall stage time sequence is obtained. The palm overall motion parameters and the fingertip local motion parameters are time-sliced according to the current collection timestamp and the stage correlation mapping rule to obtain palm overall motion parameters and fingertip local motion parameters under all time slices. The palm overall motion parameters and the fingertip local motion parameters under each time slice are input into a pre-trained action correlation model one by one to calculate the action correlation strength, and the corresponding action correlation strength under all time slices is obtained. ​ 4. The massage teaching method based on multi-modal data collaboration according to claim 3, characterized in that, The paired palm overall motion parameters and fingertip local motion parameters under all time slices after space calibration are input into a pre-trained action correlation model to calculate action correlation strength, to obtain corresponding action correlation strength under all time slices, which includes: The paired palm overall motion parameters and fingertip local motion parameters under all time slices after space calibration are input into a pre-trained action correlation model one by one, and the constraint deviations of the paired palm overall motion parameters and fingertip local motion parameters under each time slice under each type of constraint condition are calculated respectively through four types of preset constraint conditions built in the action correlation model, to obtain four types of constraint deviations under all time slices, the four types of preset constraint conditions include displacement constraint condition, velocity constraint condition, acceleration constraint condition and posture constraint condition, and the four types of constraint deviations include displacement constraint deviation, velocity constraint deviation, acceleration constraint deviation and posture constraint deviation; The displacement constraint condition is: ； wherein, represents a displacement constraint deviation, represents a tangent direction angle of a palm overall motion trajectory in the paired palm overall motion parameters at the time slice i, represents a tangent direction angle of a fingertip local motion trajectory in the paired fingertip local motion parameters at the time slice i, represents a first threshold value; The velocity constraint condition is: ； ; wherein, represents the palm overall velocity in the paired palm overall motion parameters under the time slice i, represents the fingertip local velocity in the paired fingertip local motion parameters under the time slice i, represents the velocity constraint deviation, represents the first coefficient, represents the second coefficient; The acceleration constraint condition is: ； wherein, represents an acceleration constraint deviation, represents a palm overall acceleration peak in the paired palm overall motion parameters at the time slice i, represents a fingertip local acceleration peak in the paired fingertip local motion parameters at the time slice i, represents a second threshold value; The posture constraint condition is: ； ； wherein, represents a palm overall posture angle change rate in the paired palm overall motion parameters under the time slice i, represents a fingertip local posture angle change rate in the paired fingertip local motion parameters under the time slice i, represents a third threshold value, represents a fourth threshold value, represents a posture constraint deviation; The four types of constraint deviations under all time slices are substituted into a weighting formula one by one to calculate the action correlation strength, to obtain the corresponding action correlation strength under all time slices, and the weighting formula is: ； wherein, represents the action correlation strength under the time slice i, represents the first weight, represents the second weight, represents the third weight, represents the fourth weight.

5. The massage teaching method based on multi-modal data collaboration according to claim 1, characterized in that, The fusion processing of the action-coordinated palm overall motion parameters and action-coordinated fingertip local motion parameters and the pressure parameters obtains the dual-mode fusion features, which includes: The time difference of the action-coordinated palm overall motion parameters and action-coordinated fingertip local motion parameters at a preset key node under the same time slice is calculated to obtain the time difference corresponding to each time slice, and it is judged whether the time difference corresponding to each time slice is lower than a first threshold value, if yes, the time slice with the time difference lower than the first threshold value is taken as a candidate time slice; The ratio of the overall pressure change rate in the palm overall pressure parameter to the local pressure change rate in the fingertip local pressure parameter under each candidate time slice is calculated to obtain the ratio corresponding to each candidate time slice, and it is judged whether the ratio corresponding to each candidate time slice is located in a first preset interval, if not, the candidate time slice with the ratio not located in the first preset interval is filtered to obtain an effective time slice; The fusion processing of the action-coordinated palm overall motion parameters and action-coordinated fingertip local motion parameters under the effective time slice and the pressure parameters obtains the dual-mode fusion features.

6. The massage teaching method based on multi-modal data collaboration according to claim 5, characterized in that, The fusion processing of the action-coordinated palm overall motion parameters and action-coordinated fingertip local motion parameters under the effective time slice and the pressure parameters obtains the dual-mode fusion features, which includes: According to the action-coordinated palm overall motion parameters and action-coordinated fingertip local motion parameters under the effective time slice, a preliminary hand type is identified to obtain a preliminary hand type, a matching standard pressure distribution entropy is called from a pre-constructed standard hand library according to the preliminary hand type, and the palm overall pressure distribution entropy in the palm overall pressure parameter and the fingertip local pressure distribution entropy in the fingertip local pressure parameter are integrated to obtain a current pressure distribution entropy; inputting the current pressure distribution entropy, the standard pressure distribution entropy and the effective time slice into a first synchronization rate formula to obtain a palm and fingertip motion synchronization rate, the first synchronization rate formula being: ; ； wherein, represents a motion synchronization rate of the palm and the finger tips, represents a pressure distribution entropy correction coefficient, represents a current pressure distribution entropy, represents a standard pressure distribution entropy, represents a number of valid time slices, represents a total number of time slices; integrating a palm overall average pressure value in the palm overall pressure parameter and a fingertip local average pressure value in the fingertip local pressure parameter to obtain a current average pressure value sequence, calling a matched standard speed sequence from a pre-constructed standard technique library according to the preliminary technique type, and inputting the standard speed sequence and the current average pressure value sequence into a second synchronization rate formula to obtain a force and speed force-speed synchronization rate, the second synchronization rate formula being: ； wherein, represents the force-velocity synchrony rate of pressure and velocity, represents the Pearson correlation coefficient of the current mean pressure value sequence and the standard velocity sequence, represents the current mean pressure value sequence, represents the standard velocity sequence, represents the dynamic time warping distance of the current mean pressure value sequence and the standard velocity sequence, represents the largest dynamic time warping distance in the standard technique library.

7. The massage teaching method based on multi-modal data collaboration according to claim 1, characterized in that, the double-mode fusion feature, the pressure parameter and the motion-coordinated palm overall motion parameter and the motion-coordinated fingertip local motion parameter are collectively inputted into a pre-trained massage technique recognition model as current massage parameters to perform massage technique recognition to obtain a current massage technique type, and N standard massage parameters matched with the N core parameters after dimension reduction are called from a pre-constructed standard technique library according to the current massage technique type, and a difference degree between the N standard massage parameters and the N core parameters after dimension reduction is calculated. the current massage parameters are normalized to a [0, 1] interval by using Min-Max normalization to realize standardization processing of the current massage parameters, and the current massage parameters after standardization processing are obtained; N core parameters are screened from the current massage parameters after standardization processing by using a random forest algorithm, and the N core parameters are processed by dimension reduction by using a principal component analysis algorithm to obtain the N core parameters after dimension reduction; the N core parameters after dimension reduction are inputted into the pre-trained massage technique recognition model to perform massage technique recognition to obtain the current massage technique type, wherein the pre-trained massage technique recognition model is constructed and trained based on an LSTM network; N standard massage parameters matched with the N core parameters after dimension reduction are called from the pre-constructed standard technique library according to the current massage technique type, and a difference degree between the N standard massage parameters and the N core parameters after dimension reduction is calculated.

8. The massage teaching method based on multi-modal data collaboration according to claim 7, characterized in that, the difference degree between the N standard massage parameters and the N core parameters after dimension reduction includes: a parameter type of each core parameter is obtained, when the parameter type is a numerical type, a difference formula is used to calculate a difference degree between the core parameter and a corresponding standard massage parameter; when the parameter type is a direction type, a vector included angle cosine theorem is used to calculate the difference degree between the core parameter and the corresponding standard massage parameter; when the parameter type is a time sequence type, whether the time sequence type is a periodic uniformity time sequence is judged, if yes, a variation coefficient method is used to calculate the difference degree between the core parameter and the corresponding standard massage parameter, if not, a waveform similarity algorithm is used to calculate the difference degree between the core parameter and the corresponding standard massage parameter; when the parameter type is a stability type, a standard deviation formula is used to calculate the difference degree between the core parameter and the corresponding standard massage parameter.

9. A massage teaching system based on multi-modal data collaboration, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, the processor executes the computer program to implement the method in any one of claims 1 to 8.

Images