Personalized massage prescription generation system based on multi-modal perception and semantic mapping

Abstract

The application discloses a personalized massage prescription generation system based on multi-modal perception and semantic mapping, and belongs to the technical field of medical intelligence and modernization of traditional Chinese medicine. The system comprises a multi-modal perception and feature extraction module, a semantic analysis and knowledge reasoning module, an intelligent prescription decision engine module and a self-adaptive compliant control and execution module. The method realizes the full-process intelligentization of massage treatment through four core steps of multi-modal data fusion, semantic analysis of traditional Chinese medicine chief complaint, intelligent prescription generation and closed-loop adaptive execution. The application realizes the digitization of the logic of traditional Chinese medicine syndrome differentiation and treatment through semantic mapping of "syndrome-position-method", realizes the accurate assessment of the patient's body state and physiological state through multi-modal space-time fusion, realizes safe and adaptive massage execution based on the closed-loop control of the muscle fatigue index, solves the problems of blind execution, lack of personalization and large safety hazards of traditional massage equipment, and promotes the standardized inheritance and grass-roots popularization of traditional Chinese medicine massage skills.

Description

Technical Field

[0001] This invention belongs to the field of medical intelligence and the modernization of traditional Chinese medicine, and in particular, it is a system that generates personalized intelligent massage treatment plans through multimodal data fusion and "symptom-position-method" semantic mapping logic. Background Technology

[0002] Massage, as a highly representative non-drug therapy in Traditional Chinese Medicine, has demonstrated significant and widespread clinical efficacy in relieving musculoskeletal pain, improving local blood circulation, and promoting functional rehabilitation. However, the popularization and standardization of traditional massage within the modern medical system faces severe challenges. The training period for massage therapists is lengthy, and the transmission of skills often relies on the traditional "master-apprentice" model. This results in treatment effects being highly dependent on the therapist's personal subjective experience and sensory intuition, making it difficult to quantify the so-called "feelings under the touch" through objective data.

[0003] In existing research on automated massage equipment, although some attempts have been made to execute massage movements through robotic arms or preset programs, these systems generally suffer from the problem of "blind execution." Most existing systems use fixed trajectories and preset forces, lacking precise capture of the patient's individual posture, muscle tension, and real-time physiological feedback. Clinically, different patients exhibit significant differences in their physical manifestations (such as muscle stiffness, pain threshold, and stress distribution) for the same condition. Without precise, individualized treatment, not only is it difficult to achieve ideal therapeutic effects, but it may even cause soft tissue damage.

[0004] Furthermore, the core of TCM clinical diagnosis lies in "syndrome differentiation and treatment," where patients' chief complaints (such as "distending pain," "dull pain," and "local coldness") contain important pathological information. Traditional massage robots lack the semantic understanding ability of these unstructured chief complaints, failing to deeply correlate patients' psychological sensations, pain point descriptions, and objective physical examination results. Currently, there is an urgent need for an intelligent system that can integrate depth vision, biomechanical feedback, and semantic understanding to solve industry challenges such as the difficulty in quantifying massage techniques, the strong subjectivity of prescription generation, and poor human-computer interaction security. Summary of the Invention

[0005] The present invention aims to overcome the above-mentioned defects of the prior art and specifically solves the following technical problems:

[0006] To address the problem that existing massage devices have a single sensing dimension and cannot integrate the patient's macroscopic body posture and microscopic muscle physiological state, resulting in insufficient accuracy in lesion localization and status assessment;

[0007] This addresses the problem that existing technologies cannot understand unstructured chief complaints in Traditional Chinese Medicine (TCM) and cannot digitize the logic of "syndrome differentiation and treatment," resulting in a lack of personalized prescription generation and support from TCM theory.

[0008] This addresses the problem that traditional massage equipment uses fixed-program open-loop control without a biofeedback mechanism, which can easily lead to ineffective treatment or secondary damage caused by excessive massage.

[0009] To address the challenges of standardizing and digitizing massage techniques, and the difficulty of bringing high-quality medical resources to grassroots levels.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] A personalized massage prescription generation system based on multimodal perception and semantic mapping includes multimodal data acquisition hardware, a robotic arm actuator, and a processor electrically connected to both. The processor is equipped with a multimodal perception and feature extraction module, a semantic parsing and knowledge reasoning module, an intelligent prescription decision engine module, and an adaptive compliant control and execution module that interact sequentially.

[0012] The system comprises the following modules: a multimodal perception and feature extraction module, used to collect and fuse patients' visual posture data and muscle physiological data to generate spatiotemporally aligned multimodal fusion features; a semantic parsing and knowledge reasoning module, used to obtain patients' chief complaint information, and based on a pre-constructed TCM massage "syndrome-position-method" knowledge graph, used to perform semantic parsing and logical reasoning on the chief complaint information to generate syndrome differentiation semantic feature vectors; an intelligent prescription decision engine module, used to receive multimodal fusion features and syndrome differentiation semantic feature vectors, generate five-dimensional personalized massage prescription vectors, and map them into motion trajectory instructions that can be executed by the robotic arm; and an adaptive compliant control and execution module, used to drive the robotic arm to perform massage operations and complete adaptive adjustments and safety management based on real-time biofeedback data.

[0013] Furthermore, the multimodal perception and feature extraction module includes a visual pose acquisition unit, a physiological signal monitoring unit, and a spatiotemporal registration and fusion unit, realizing the spatiotemporal alignment and weighted fusion of visual geometric information and electromyographic physiological information.

[0014] Furthermore, the semantic parsing and knowledge reasoning module includes a multimodal human-computer interaction unit, a semantic entity extraction unit, and a graph reasoning engine unit. Through large language models and graph retrieval-enhanced generation technology, it realizes semantic mapping from unstructured complaints to structured dialectical results.

[0015] Furthermore, the intelligent prescription decision engine module includes a multi-task prediction network unit and a digital technique generation library unit. It realizes the automatic generation of prescription parameters through an end-to-end deep learning model and maps standardized technique templates to the patient's individual coordinate system.

[0016] Furthermore, the adaptive compliant control and execution module includes an impedance control execution unit, a therapeutic effect assessment and feedback unit, and a safety abnormality circuit breaker unit, thus constructing a treatment endpoint determination mechanism based on the muscle near fatigue index and a full-process safety protection system.

[0017] On the other hand, the present invention provides a method for generating personalized massage prescriptions based on multimodal perception and semantic mapping, comprising the following steps:

[0018] S1 Multimodal Perception and Spatial Mapping: Collects patients' visual posture data and muscle physiological data, completes the spatiotemporal alignment and feature fusion of heterogeneous data, and generates multimodal fusion features;

[0019] S2 Chief Complaint Semantic Parsing and Graph Reasoning: Collect patients' chief complaint information, complete medical entity extraction, and complete dialectical reasoning based on the pre-constructed TCM massage "syndrome-position-method" knowledge graph to generate dialectical semantic feature vectors;

[0020] S3 Personalized Prescription Decision Generation: Input multimodal fusion features and dialectical semantic feature vectors into a pre-trained multi-task deep learning model to generate a five-dimensional personalized massage prescription vector, and map it into a motion trajectory command that can be executed by the robotic arm;

[0021] S4 Adaptive Compliant Control and Closed-Loop Execution: Drives the robotic arm to perform massage operations according to motion trajectory instructions, and completes adaptive adjustment of force, automatic determination of treatment endpoint, and full-process safety management based on real-time biofeedback data.

[0022] This technology proposes a personalized massage prescription generation system based on multimodal perception and semantic mapping, which has the following advantages and beneficial effects:

[0023] 1. This invention overcomes the limitations of traditional massage robots' "blind execution," achieving precise treatment based on deep physiological states. Through multimodal perception and spatiotemporal fusion technology, it spatiotemporally aligns macroscopic skeletal posture captured by 3D vision with microscopic muscle physiological signals acquired by surface electromyography. This not only allows the invention to "see" abnormalities in the patient's body structure but also to "perceive" high-tension and fatigue points within the muscles. Experimental data shows that its accuracy in assessing muscle state is approximately 35% higher than a single visual approach, achieving a qualitative leap from "contour-based massage" to "lesion-based massage."

[0024] 2. This invention solves the problem of unstructured TCM chief complaints being unquantifiable, realizing the digital implementation of the "syndrome differentiation and treatment" logic. It constructs a three-dimensional semantic mapping architecture of "syndrome-position-method," using a large language model and graph retrieval-enhanced generation technology to map patients' colloquial chief complaints into standardized TCM syndrome differentiation entities, and infers prescription parameters that conform to TCM theory. This improves the F1 score for chief complaint comprehension to over 90%, endowing the massage robot with the syndrome differentiation thinking ability of TCM experts and solving the problem of monotonous prescriptions in traditional equipment.

[0025] 3. This invention eliminates the safety risks of open-loop control and establishes an adaptive closed-loop treatment system based on biofeedback. It introduces the muscle near-fatigue index as the core indicator for determining the treatment endpoint, achieving "on-demand treatment" and avoiding secondary damage caused by excessive massage. Simultaneously, combined with an impedance control algorithm optimized by reinforcement learning, it can execute a yielding action in milliseconds when defensive muscle contractions are detected, significantly improving the safety and comfort of human-computer interaction.

[0026] 4. Promotes the standardization and digital inheritance of massage techniques. This invention, by constructing a digital technique library and a "certificate-position-method" knowledge graph, transforms the experience of renowned massage masters into recordable and reproducible algorithm parameters, breaking down the barriers of "master-apprentice" inheritance. It provides quantitative data support for the scientific research of traditional Chinese massage, is easy to promote in grassroots medical institutions and rehabilitation centers, and has extremely high clinical application and social value. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall structure of the system of the present invention;

[0028] Figure 2 This is a schematic diagram of the present invention. Detailed Implementation

[0029] The present invention will be further described below with reference to the embodiments. It should be noted that these are merely examples and descriptions of the inventive concept. Those skilled in the art can make various modifications or additions to the specific embodiments described or use similar methods to replace them, as long as they do not deviate from the inventive concept or exceed the scope defined in the claims, they should all be considered to fall within the protection scope of the present invention.

[0030] like Figures 1-2 As shown, the personalized massage prescription generation system based on multimodal perception and semantic mapping proposed in this invention includes: a multimodal perception and feature extraction module, a semantic parsing and knowledge reasoning module, an intelligent prescription decision engine module, and an adaptive compliant control and execution module.

[0031] 1. Multimodal perception and feature extraction module;

[0032] This module serves as the system's data entry point, responsible for converting analog signals from the physical world into digital feature vectors that can be processed by a computer. It specifically comprises the following three sub-units:

[0033] Visual pose acquisition unit:

[0034] Function: Equipped with a 3D depth vision sensor (such as a ToF camera) to acquire real-time depth point cloud data streams of the patient's body surface. Processing Logic: Internally integrates pose estimation algorithms (such as an improved CNN model) to identify spinal keypoints (C1-L5), pelvic tilt angles, and limb kinematic chain coordinates from the point cloud data. Output: Generates a visual feature tensor containing human geometric information. This tensor describes the patient's macroscopic body structure and skeletal location.

[0035] Physiological signal monitoring unit:

[0036] Function: Equipped with a multi-channel surface electromyography (sEMG) sensor array and an infrared thermal imaging sensor, it is used to acquire muscle physiological electrical signals and body surface temperature distribution. Processing Logic: Includes signal preprocessing circuitry (filtering, amplification) and feature extraction algorithms. This unit calculates the root mean square value of the sEMG signal through short-time Fourier transform (STFT). ) and median frequency ( The system utilizes infrared data to generate a temperature gradient matrix. Output: A physiological feature vector reflecting muscle tension, fatigue, and inflammation. .

[0037] Spatiotemporal registration and fusion unit:

[0038] Function: Resolves the heterogeneity issue between visual spatial data and electromyographic temporal data. Processing logic: Utilizes a hand-eye calibration matrix to establish the visual coordinate system. Mapped to the robot arm coordinate system Simultaneously, a dual-stream network architecture and attention mechanism are employed to dynamically calculate the fusion weights based on the confidence level of each modality's data (such as visual occlusion or EMG signal-to-noise ratio). Connection relationship: This unit receives raw data from the visual pose acquisition unit and the physiological signal monitoring unit, outputs aligned multimodal fusion features, and transmits them to the intelligent prescription decision engine module.

[0039] 2. Semantic parsing and knowledge reasoning module;

[0040] This module serves as the system's "cognitive center," responsible for simulating the consultation logic of a traditional Chinese medicine expert, transforming unstructured patient complaints into structured reasoning results. Specifically, it includes:

[0041] Multimodal human-computer interaction unit:

[0042] Function: Provides voice acquisition (microphone array) and text input interface for obtaining patients' subjective medical history descriptions (chief complaints). Processing logic: Integrates an automatic speech recognition (ASR) engine to transcribe speech streams into text data and perform preliminary noise reduction and word segmentation.

[0043] Semantic entity extraction unit:

[0044] Functionality: Based on Large Language Model (LLM) and Named Entity Recognition (NER) technologies, this program performs deep structured parsing of text. Processing Logic: It identifies and extracts three core entities from the text: symptom entities (such as "distending pain"), location entities (such as "Dazhui acupoint"), and degree / nature entities (such as "intense" and "pulling sensation").

[0045] Graph Inference Engine Unit:

[0046] Function: Performs logical reasoning based on a pre-built "Syndrome-Location-Method" TCM knowledge graph. Processing Logic: This unit stores an association rule graph covering meridians, acupoints, pathogenesis, and techniques. Utilizing Graph Retrieval Augmentation (GraphRAG) technology, it performs path search based on the input entity nodes. For example, it derives "Syndrome" nodes from "Symptom" nodes, and then associates the "Syndrome" nodes with the optimal "Treatment Acupoints" and "Techniques" nodes. Connection Relationships: This module outputs a semantic feature vector containing the syndrome differentiation results, recommended acupoints, and suggested techniques. The data is then transmitted to the intelligent prescription decision engine module.

[0047] 3. Intelligent prescription decision engine module;

[0048] This module is the core decision-making body of the system, responsible for integrating the physical data of the perception layer with the cognitive data of the semantic layer to generate specific execution instructions.

[0049] Multi-task prediction network unit:

[0050] Function: Generates personalized massage prescriptions based on a deep learning model. Processing logic: Employs an end-to-end non-linear mapping function. This unit receives data from the sensing module. and from semantic modules The five-dimensional prescription parameters are predicted by multilayer perceptron (MLP): technique type, treatment site coordinates, target force value, operation frequency, and duration.

[0051] Digital Technique Generation Library Unit:

[0052] Function: Stores standardized massage motion trajectory data. Processing Logic: Includes a built-in "Master Massage Technique Model," containing dynamic feature templates for various techniques such as rolling, kneading, and grasping. Based on the "technique type" parameter output by the prediction network, this unit calls the corresponding trajectory template and combines it with the "treatment area coordinates" to generate the Cartesian space motion path of the robotic arm's end effector. Connection: This module sends the generated complete prescription instructions (trajectory point set and force control parameters) to the adaptive compliant control and execution module.

[0053] 4. Adaptive compliant control and execution module;

[0054] This module is the system's execution mechanism, responsible for driving the hardware and ensuring safety and comfort during execution.

[0055] Impedance control execution unit:

[0056] Function: Drives the joint motors of the robotic arm to perform massage movements and achieve compliant interaction. Processing Logic: Employs an impedance controller optimized based on reinforcement learning (PPO algorithm). This unit receives feedback data from the end effector's six-dimensional force sensor in real time. When it detects that the contact force exceeds a set threshold or there is a sudden change in tissue stiffness (such as muscle spasm), it automatically adjusts the impedance parameters (stiffness and damping) of the robotic arm and performs a "yielding" action to avoid hard resistance.

[0057] Treatment efficacy assessment and feedback unit:

[0058] Function: Real-time assessment of treatment effectiveness and decision on whether to terminate the procedure. Processing logic: Continuously calculates the muscle near fatigue index (PFI), using the following formula: This unit monitors The value changes, and when it rises to a preset threshold (such as...) When the system determines that the muscles in the current area have been relaxed, it sends a "task completed" signal to the decision engine, triggering a change of acupoint or a shutdown command.

[0059] Safety Fault Fuse Unit:

[0060] Function: Provides the highest priority safety protection. Processing Logic: Utilizes the Isolation Forest algorithm to monitor the entire system's data flow. If a high-frequency burst (convulsion) of sEMG signal or abnormal visual positioning deviation is detected, the motor power is immediately cut off and the robotic arm is locked to ensure absolute patient safety. Connectivity: This module is directly connected to all sensors and actuators, forming a low-level hardware safety loop.

[0061] The specific working method of the system of the present invention includes the following steps:

[0062] Step 1: Semantic Mapping-Based Chief Complaint Parsing and Graph Reasoning

[0063] Before treatment begins, the system collects the patient's spoken complaints through the voice interaction module.

[0064] 1.1 Entity Extraction and Structuring: The system utilizes a built-in Large Language Model (LLM) to perform Natural Language Processing (NLP) on the speech-transcribed text. Through Named Entity Recognition (NER) technology, the following key entities are extracted: Symptom Entity. "Distending pain" (nature: excess syndrome), "stiffness" (nature: muscle tension); location: solid ( "Dazhui acupoint", "back of the neck"; causative entity ( "Prolonged head-down posture" (suggests chronic strain).

[0065] 1.2 "Syndrome-Location-Method" Graph Reasoning: The system calls a pre-constructed TCM massage knowledge graph and uses GraphRAG technology for logical deduction: Syndrome: Based on "distending pain" + "stiffness," combined with graph rules, the pathological essence is determined to be "cervical spondylosis" or "neck and back myofascitis." Location: Using "Dazhui" acupoint as the anchor point, its associated meridian (Du Mai) is retrieved, and related acupoints are automatically expanded to determine the treatment target area as the area connecting Dazhui, Fengchi, and Jianjing acupoints. Manipulation: Based on "distending pain" (requiring purging method) and "stiffness" (requiring softening method), the optimal combination of manipulation techniques is matched: rolling method (relaxation) and acupressure method (analgesia). Output: Generates an initial semantic feature vector. .

[0066] Step 2: Spatiotemporal alignment and feature fusion of multimodal data

[0067] The system activates the sensing devices to perform a physical-level digital scan of the patient.

[0068] 2.1 Visual Pose Feature Extraction: Point cloud data captured by a 3D camera was denoised, and a pose estimation algorithm was used to identify the straightening of the physiological curvature of the cervical spine (C1-C7) and calculate the scoliosis angle. .

[0069] 2.2 Physiological Feature Extraction: sEMG sensors acquire trapezius muscle signals. The system uses a sliding window (WindowSize= ) Calculate the root mean square value of the signal ( ) and median frequency ( In this example, data shows that the right trapezius muscle... The value was significantly higher than that on the left, indicating the presence of a high-tension cord-like nodule on the right.

[0070] 2.3 Spatiotemporal Coordinate Alignment: Since visual data is located in the camera coordinate system The electromyography (EMG) data is located in the local coordinate system of the patch, and the system uses a hand-eye calibration matrix. Unify all data to the robot arm base coordinate system The conversion formula is:

[0071]

[0072] in, For rotation matrix, This is the translation vector. Therefore, the system accurately maps the "high-tension points" detected by electromyography into the motion space of the robotic arm.

[0073] 2.4 Feature Fusion: A two-stream neural network is used to fuse visual features. With electromyographic characteristics The algorithm calculates weights using an attention layer mechanism. Since electromyographic abnormalities (high tension) are more significant than visual abnormalities (curvature) in this example, the algorithm automatically assigns higher weights to electromyographic features. .

[0074] Step 3: Personalized Prescription Decision Generation

[0075] The prescription decision engine receives the fused data and generates the final executable instructions.

[0076] 3.1 Prescription Vector Calculation: Using a trained multi-task deep learning model, prescriptions are generated through an end-to-end mapping function.

[0077]

[0078] Output prescription vector .

[0079] 3.2 Specific prescription parameters: For this patient, the generated prescription is as follows:

[0080] Phase 1 (Release): Upper trapezius muscle on the right side; Technique: Rolling; Intensity: (Adjusted based on BMI); Frequency: Duration: .

[0081] Phase Two (Pain Relief): Dazhui acupoint; Technique: Acupressure; Pressure: Duration: .

[0082] Step 4: Adaptive Execution and Control Based on Biofeedback

[0083] The robotic arm executes the massage according to the generated trajectory instructions, and a closed-loop feedback mechanism is introduced to ensure safety and therapeutic effect.

[0084] 4.1 Compliant Resistance Control: When performing the "point pressure method," if the patient experiences involuntary muscle contraction due to pain (increased counteracting torque), the six-dimensional force sensor detects the contact force. Exceeding the safety threshold The controller immediately adjusts the robotic arm's impedance parameters based on a reinforcement learning strategy (PPO algorithm), reduces stiffness, and performs a slight "yield" action to avoid causing soft tissue contusion.

[0085] 4.2 PFI-based efficacy assessment and termination: During the massage, the system continuously calculates the muscle near fatigue index (PFI) to determine the treatment endpoint.

[0086]

[0087] Initial state: At the start of treatment, The value is relatively high. This indicates muscle stiffness.

[0088] Process changes: with After minutes of rolling exercises, the muscles gradually relax, local metabolic products accumulate, and the sEMG spectrum shifts to lower frequencies. The value decreased.

[0089] Termination determination: when The value rose to When this occurs, it indicates that the muscles have fully relaxed and entered the critical point of metabolic fatigue.

[0090] Action execution: The system determines that the treatment of the area has reached the target level, automatically issues an instruction to stop the current operation, and drives the robotic arm to move to the next acupoint (such as Jianjing acupoint), thereby realizing "treatment on demand" and preventing excessive massage.

[0091] Step 5: Anomaly Monitoring and Safety Circuit Breaker

[0092] Throughout the entire process, the backend employs the Isolation Forest algorithm to monitor the data stream in real time. If abnormally high-frequency bursts of sEMG (indicating muscle spasms) or uninstructed displacements in visual positioning are detected, the system will... Internally triggered hardware-level emergency stop, locking the robotic arm joints to ensure absolute patient safety.

[0093] The above is an exemplary description of the invention. Obviously, the specific implementation of the invention is not limited to the above-described manner. Any non-substantial improvement made using the inventive concept and technical solution of the invention, or the direct application of the inventive concept and technical solution to other situations without modification, is within the protection scope of the invention.

Claims

1. A personalized massage prescription generation system based on multimodal perception and semantic mapping, comprising multimodal data acquisition hardware, a robotic arm actuator, and a processor electrically connected to the multimodal data acquisition hardware and the robotic arm actuator, characterized in that, The processor is equipped with: The multimodal perception and feature extraction module is used to collect and fuse patients' visual posture data and muscle physiological data to generate spatiotemporally aligned multimodal fusion features; The semantic parsing and knowledge reasoning module is used to obtain the patient's chief complaint information. Based on the pre-built TCM massage syndrome-position-method knowledge graph, it performs semantic parsing and logical reasoning on the chief complaint information to generate a syndrome differentiation semantic feature vector. The intelligent prescription decision engine module is electrically connected to the multimodal perception and feature extraction module and the semantic parsing and knowledge reasoning module, respectively. It is used to receive the multimodal fusion features and dialectical semantic feature vectors, generate a five-dimensional personalized massage prescription vector containing the type of technique, treatment site, target intensity, operation frequency, and duration of action, and map it into a Cartesian space motion trajectory instruction that can be executed by the robotic arm. The adaptive compliant control and execution module is electrically connected to the intelligent prescription decision engine module, the robotic arm execution mechanism, and the multimodal data acquisition hardware, respectively. It is used to drive the robotic arm to perform massage operations according to the motion trajectory instructions, and to adaptively adjust the massage operations and manage the entire process safety based on the real-time collected biofeedback data.

2. The personalized massage prescription generation system based on multimodal perception and semantic mapping according to claim 1, characterized in that, The multimodal perception and feature extraction module includes a visual pose acquisition unit, a physiological signal monitoring unit, and a spatiotemporal registration and fusion unit; The visual pose acquisition unit is equipped with a 3D depth vision sensor to acquire depth point cloud data of the patient's body surface. It extracts the set of coordinates of key points of human skeleton through an improved convolutional neural network and long short-term memory network fusion model, calculates the physiological curvature value of the spine and the height difference of the bones on both sides, and generates a visual feature tensor containing human geometric information. The physiological signal monitoring unit is equipped with a multi-channel surface electromyography sensor array and an infrared thermal imager to collect surface electromyography signals and body surface temperature distribution data of the patient's treatment area. After filtering and denoising the original signals, the root mean square value representing muscle activation intensity, the median frequency representing muscle fatigue, and the temperature gradient matrix reflecting inflammation / ischemia are extracted to generate a muscle physiological feature vector. The spatiotemporal registration and fusion unit is electrically connected to the visual posture acquisition unit and the physiological signal monitoring unit, respectively. It is used to complete the transformation between the visual sensor coordinate system and the robotic arm base coordinate system through the hand-eye calibration algorithm, realize the spatiotemporal alignment of the visual feature tensor and the physiological feature vector, and dynamically allocate fusion weights according to the confidence of each modality data through a two-stream neural network architecture with attention mechanism, complete the multimodal feature fusion, and output the aligned multimodal fused features.

3. The personalized massage prescription generation system based on multimodal perception and semantic mapping according to claim 2, characterized in that, In the spatiotemporal registration and fusion unit, the formula for converting visual feature points into coordinates in the robotic arm coordinate system is: in, These are the coordinates of the feature points in the robot arm's base coordinate system. R represents the coordinates of the feature point in the visual sensor coordinate system, R is a 3×3 rotation matrix, and T is a 3×1 translation vector.

4. The personalized massage prescription generation system based on multimodal perception and semantic mapping according to claim 1, characterized in that, The semantic parsing and knowledge reasoning module includes a multimodal human-computer interaction unit, a semantic entity extraction unit, and a graph reasoning engine unit; The multimodal human-computer interaction unit is equipped with a microphone array and a text input interface, and integrates an automatic speech recognition engine to collect patients' spoken complaints and transcribe them into structured text data. The semantic entity extraction unit is electrically connected to the multimodal human-computer interaction unit. Based on the large language model and named entity recognition technology, it extracts three types of core medical entities from the transcribed text: etiology / cause entities, location entities, and symptom / nature entities. The graph reasoning engine unit is electrically connected to the semantic entity extraction unit and has a built-in pre-constructed TCM massage "syndrome-location-method" knowledge graph. The knowledge graph contains a set of nodes and edge sets of meridian relationships and treatment method relationships for acupoints, meridians, diseases, and techniques. The graph reasoning engine unit uses graph retrieval enhancement generation technology to complete the logical reasoning of disease syndrome differentiation, treatment target location, and appropriate technique matching based on the extracted core medical entities, and outputs a syndrome differentiation semantic feature vector containing the syndrome differentiation results, recommended acupoints, and appropriate techniques.

5. The personalized massage prescription generation system based on multimodal perception and semantic mapping according to claim 1, characterized in that, The intelligent prescription decision engine module includes a multi-task prediction network unit and a digital technique generation library unit. The multi-task prediction network unit adopts a multi-task deep learning architecture and incorporates an end-to-end nonlinear mapping function: Where P is the output five-dimensional massage prescription vector. For visual feature tensors, This represents the muscle physiological feature vector. Let f be the dialectical semantic feature vector, and f be the pre-trained neural network mapping function; The multi-task prediction network unit is used to receive multimodal fusion features and dialectical semantic feature vectors, and output a five-dimensional personalized massage prescription vector; the digital manipulation generation library unit is electrically connected to the multi-task prediction network unit, and has a built-in standardized massage motion dynamic trajectory template library, which is used to call the corresponding trajectory template according to the manipulation type in the prescription vector, and combine it with the coordinates of the robotic arm coordinate system of the treatment site to generate a Cartesian space motion trajectory instruction that can be executed by the robotic arm end.

6. The personalized massage prescription generation system based on multimodal perception and semantic mapping according to claim 1, characterized in that, The adaptive compliant control and execution module includes an impedance control execution unit, a efficacy assessment and feedback unit, and a safety abnormality circuit breaker unit; The impedance control execution unit is electrically connected to the intelligent prescription decision engine module and the robotic arm execution mechanism, respectively. It adopts an impedance controller optimized based on the proximal strategy optimization algorithm to receive motion trajectory instructions to drive the robotic arm to perform massage operations, and transmits contact force data in real time through a six-dimensional force sensor at the end of the robotic arm. When the contact force exceeds the threshold or the tissue stiffness changes abruptly, the impedance parameters of the robotic arm are automatically adjusted to perform a yielding action. The therapeutic effect assessment and feedback unit is electrically connected to the impedance control execution unit and the multimodal perception and feature extraction module, respectively. It is used to continuously collect surface electromyography signals during massage, calculate the muscle fatigue index, and when the muscle fatigue index rises to a preset threshold, it determines that the treatment of the current part has reached the standard, sends a task completion signal to the intelligent prescription decision engine module, and triggers acupoint switching or shutdown command. The safety anomaly circuit breaker unit is directly connected to the hardware of all system sensors and actuators. It uses an isolated forest anomaly detection algorithm to monitor the entire process data flow in real time. When an abnormal high-frequency burst of electromyographic signal or a large non-instructional displacement of the patient's body is detected, it triggers hardware lock-up and emergency stop commands within milliseconds.

7. The personalized massage prescription generation system based on multimodal perception and semantic mapping according to claim 6, characterized in that, In the therapeutic effect assessment and feedback unit, the formula for calculating the muscle near-fatigue index is as follows: in, The muscle near-fatigue index has a value range of [0,1]. The median frequency of electromyography at the initial moment of treatment; The median frequency of electromyography at the current time t; This represents the weighting coefficient for the time window.

8. The personalized massage prescription generation system based on multimodal perception and semantic mapping according to claim 1, characterized in that, In the aforementioned syndrome-position-method knowledge graph, syndrome refers to the TCM syndrome differentiation result obtained through semantic parsing, position refers to the anatomical target point and meridian acupoint coordinates for treatment, and method refers to the set of massage techniques and dynamic parameters.

9. A method for generating personalized massage prescriptions based on multimodal perception and semantic mapping, characterized in that, The system according to any one of claims 1-8 includes the following steps: S1 Multimodal Perception and Spatial Mapping: Collects patients' visual posture data and muscle physiological data, completes the spatiotemporal alignment and feature fusion of heterogeneous data, and generates multimodal fusion features; S2 Chief Complaint Semantic Parsing and Graph Reasoning: Collect patients' chief complaint information, complete medical entity extraction, and complete dialectical reasoning based on a pre-constructed TCM massage syndrome-position-method knowledge graph to generate dialectical semantic feature vectors; S3 Personalized Prescription Decision Generation: Input multimodal fusion features and dialectical semantic feature vectors into a pre-trained multi-task deep learning model to generate a five-dimensional personalized massage prescription vector, and map it into a motion trajectory command that can be executed by the robotic arm; S4 Adaptive Compliant Control and Closed-Loop Execution: Drives the robotic arm to perform massage operations according to motion trajectory instructions, and completes adaptive adjustment of force, automatic determination of treatment endpoint, and full-process safety management based on real-time biofeedback data.

10. The personalized massage prescription generation method based on multimodal perception and semantic mapping according to claim 9, characterized in that, Step S1 specifically includes: The S11 uses a 3D depth vision sensor to acquire depth point cloud data of the patient's body surface, extracts the coordinates of key points of the human skeleton, and generates a visual feature tensor. S12 collects electromyographic signals and temperature data of the patient's treatment area through a surface electromyography sensor array and an infrared thermal imager, extracts muscle activation intensity, fatigue and inflammation status characteristics, and generates muscle physiological feature vectors. S13 completes the transformation between the visual coordinate system and the robotic arm coordinate system through the hand-eye calibration algorithm, realizes the spatiotemporal alignment of visual features and physiological features, and completes multimodal feature fusion through a two-stream neural network with attention mechanism, outputting multimodal fused features; Step S2 specifically includes: S21 collects patients' spoken complaints through voice or text interaction, and completes speech transcription and text preprocessing. S22 is based on a large language model and named entity recognition technology to extract three core medical entities from the chief complaint text: etiology / cause entity, location entity, and symptom / nature entity. S23 utilizes graph retrieval-enhanced generation technology to complete logical reasoning of disease and syndrome differentiation, treatment target location, and appropriate method matching in the "syndrome-position-method" knowledge graph based on core medical entities, generating a syndrome differentiation semantic feature vector. Step S4 specifically includes: The S41 uses an impedance control strategy optimized by a proximal strategy optimization algorithm to drive the robotic arm to perform massage operations. It collects contact force data in real time and automatically performs a retreat action when the contact force exceeds the threshold or when the muscle defensively contracts. S42 continuously collects electromyographic signals and calculates the muscle near-fatigue index. When the index rises to a preset threshold, it determines that the treatment of the current part has reached the target and automatically switches the treatment part or terminates the treatment. S43 uses the isolated forest algorithm to monitor the entire data flow in real time. When an abnormal state is detected, it immediately triggers a hardware emergency stop to complete safety management.

Images