AI robot system and method for treating tumor edema through end-to-end learning expert manipulation

Abstract

The invention discloses an AI robot system and method for treating tumor edema through an end-to-end learning expert technique, and belongs to the technical field of medical robots. The system comprises an overlook bed 3D camera, a humanoid robot hand with a combined training glove, a multi-degree-of-freedom humanoid robot arm, a robot 3D camera, a main control computer, a mobile chassis, an intelligent terminal, application software and a cloud server data center. The method comprises the steps of collecting manipulation treatment training data of a user and uploading the data to a cloud; generating matched expert-level treatment strength, speed and rhythm data by using a cloud expert manipulation large model or an expert manipulation database, fusing user training paths and postures, synthesizing new treatment data, downloading the new treatment data to a system for execution, and detecting human body movement by using a visual algorithm to carry out following treatment; according to the invention, the expert-level manipulation treatment of tumor edema can be realized, the treatment efficiency can be improved by several times, and the treatment fatigue can be relieved.

Description

Technical Field

[0001] This invention relates to the field of medical robot technology, and in particular to an AI robot system and method for end-to-end learning of expert techniques to treat tumor edema. Background Technology

[0002] Cancer patients often develop lymphedema (hereinafter referred to as tumor edema) due to damage to the lymphatic system caused by surgery, radiotherapy, and chemotherapy. This accounts for a major portion of lymphedema cases and is a painful, disabling chronic disease that is difficult to cure and requires manual lymphatic drainage and pressure dressings for control. Currently, there are over 250 million patients worldwide, with tens of millions new cases annually. In my country, the number has reached tens of millions, primarily due to damage from cancer treatment, with a higher incidence in patients after breast cancer and gynecological cancer surgery. The internationally recognized standard treatment for lymphedema is Complete Decongestive Therapy (CDT). This therapy typically involves manual lymphatic drainage (MLD) performed by a professional physician in a hospital for approximately 60 minutes daily for 20–40 days. After treatment, patients still need to perform manual drainage at home for approximately 30 minutes daily to maintain the therapeutic effect and prevent edema progression.

[0003] The industry generally recognizes that manual lymphatic drainage is the most effective treatment for edema, superior to most instrumental treatments. However, it has significant limitations: each manual treatment by a doctor is labor-intensive and time-consuming, making it difficult to maintain consistent intensity and strength throughout the 1-hour treatment session; and it is difficult for patients to maintain edema through daily self-manual treatment at home, making it difficult to persist and potentially leading to a worsening of the condition.

[0004] To alleviate the workload of doctors and the shortage of treatment resources, several auxiliary therapeutic devices have emerged, such as negative pressure drainage devices, air pressure gradient pumps, and electric massage devices. However, these devices still have significant limitations in clinical application. For example, a C-shaped airbag device disclosed in Chinese Patent CN109771246B, while providing local pressure, cannot achieve the complex techniques and dynamic adjustment capabilities required by human hands in lymphatic drainage; the multi-module massage mechanism described in Chinese Patent CN111407635B, while having some adaptability to different body parts, has a fixed movement trajectory and lacks real-time pressure sensing and path planning capabilities; and Chinese Patent CN119112630B provides a portable therapeutic device, which improves ease of use, but still fails to achieve precise pressure control and the reproduction of expert techniques.

[0005] The existing auxiliary treatment devices for lymphedema mainly have the following technical problems:

[0006] 1) It does not mimic the changes in the doctor's hand and lymphatic drainage techniques in the treatment of edema: Most existing equipment uses contact massage heads with fixed shapes (such as flat heads, ball heads, and cone heads), which cannot adapt to the irregular edema surface treatment by changing hand shape and posture like human hands. In addition, the fixed programming control of the massage head makes it difficult to replicate the therapeutic effect of doctor's lymphatic drainage techniques, and it is also impossible to adjust the appropriate contact posture and intensity according to the different edema sites of patients.

[0007] 2) Weak adaptability of force control: Due to the large changes in the irregular curvature and elasticity of the human body surface, existing equipment is difficult to achieve the same range of force control as expert manual techniques. The standard requires control within the range of 20-30 mmHg. Too high pressure can easily cause lymphatic vessel damage, while too low pressure will result in poor treatment effect.

[0008] 3) Limited adaptability to different sites: Existing auxiliary treatment devices cannot be fully applied to different sites where edema often occurs, such as the limbs, trunk, head and neck, which limits their clinical applicability and treatment effect. Summary of the Invention

[0009] (I) Purpose of the Invention

[0010] This invention aims to address the pain points of long working hours, high fatigue, and low treatment efficiency of physicians in the treatment of tumor edema, the difficulty of patients operating the device independently, and the technical problem that existing auxiliary treatment devices cannot learn and reproduce expert-level treatment techniques. It provides an AI robot system and method that learns from expert physicians' treatment techniques for tumor edema through multimodal data perception. By using expert treatment technique training data, it learns and reproduces expert physicians' lymphatic drainage techniques, achieving digital, intelligent, automated, standardized, and networked management of the treatment process, significantly improving the accuracy, accessibility, and efficiency of treatment.

[0011] (II) Technical Solution

[0012] To achieve the purpose of the invention and solve the current problems in this industry, the present invention provides an AI robot system and method for end-to-end learning of expert techniques to treat tumor edema. The robot system includes: a 3D camera looking down at the bed, a humanoid robotic hand with a fitted training glove, a multi-degree-of-freedom humanoid robotic arm, a robot 3D camera, a main control computer, a mobile chassis, a smart terminal and application software, and a cloud server data center.

[0013] The overhead 3D camera is connected wirelessly or wired to the robot system's main control computer. It is controlled by a machine vision real-time detection and recognition algorithm running on the main control computer. It is fixed in a suitable position above the middle of the bed by a headboard bracket to ensure that the imaging field of view covers the bed, the patient, and the robot's working area on the side of the bed. It is used to collect three-dimensional image data of the patient's body on the bed, detect the patient's body position movement in real time, and calibrate the bed position, the patient's position, and the robot's working position. It can also collect short video data of the expert manual therapy training process, identify the position and movement of the patient's body surface feature points in real time, and dynamically update the treatment path coordinates when a change in the patient's body position is detected, so as to realize the adaptive following of the robot's end effector to the patient's body position.

[0014] The humanoid robotic hand with a fitted training glove is designed to mimic human hand operations. It uses hydrogel material to mimic muscles and skin, creating a palm and fingertips that closely resemble the appearance, muscle elasticity, and feel of a human hand. The palm integrates a vibration motor module and a heating module to enhance the treatment experience. Thin-film dot-matrix tactile sensors are arranged on the palm and finger surfaces to monitor the distribution of contact force in real time. The training glove incorporates sensors for the bending angles of each finger joint, capturing the trainee's finger movements and postures in real time. This data is then transmitted synchronously to the main control computer via wired or wireless means, driving the robotic hand to perform grasping, pressing, and other operations identical to a human hand. The back of the training glove features touch control buttons, including on / off, training, delete, and treatment buttons, used to switch the robot's communication control training status, start and stop training data acquisition, delete training data, and trigger automatic treatment mode, respectively.

[0015] The multi-degree-of-freedom humanoid robotic arm: its end is connected to the humanoid robotic hand through a six-dimensional force sensor and a flange, and has high-precision trajectory, posture and force control capabilities, supporting the reproduction and execution of complex manual movements through teaching and training;

[0016] The robot's 3D camera is mounted on a horizontal rotating joint on the robot arm base and is used for recording video of the manual therapy process from a horizontal perspective and for safety monitoring.

[0017] The main control computer integrates multimodal data acquisition and storage, network communication, and scheduling control functions for the robot to perform manual therapy. After each doctor trains the basic operation of the treatment technique, the robot can directly reproduce it multiple times. The strength, speed, and rhythm of the control process can be set and adjusted. The application software of the smart terminal can be set to automatically connect to the cloud server or connect to the cloud server through the application software to upload doctor treatment data and expert treatment data to the cloud server data center. The current doctor's manual therapy path and manual posture data are retained. By selecting any expert's strength, speed, and rhythm in the expert manual therapy database in the cloud server data center that matches the patient's edema state, the strength control level of that expert is integrated to generate new treatment data. Alternatively, a generative expert strength, speed, and rhythm that matches the patient's edema state can be generated through a generative pre-training model of expert manual therapy. New treatment data is generated by training the large model through multiple expert manual therapy methods. The smart terminal downloads the data to the main control computer and controls the robot to realize expert-level manual therapy functions.

[0018] The mobile chassis includes a manually pushable or remotely controlled electric mobile navigation function. The chassis is equipped with an electric negative pressure suction cup to enhance ground fixation and integrates a rechargeable power battery to provide the system with electric drive mobility and emergency power supply.

[0019] The intelligent terminal and application software: The intelligent terminal includes a smartphone, tablet, or general-purpose computer. The application software runs on the intelligent terminal device, providing human-computer interaction management parameters, supporting experts and doctors to set treatment parameters, manage data, download expert data from the cloud, and monitor the treatment process in real time. The application software also integrates: AI expert doctor voice interaction function, integrating a rehabilitation professional knowledge base and a patient psychology knowledge base, applying the retrieval enhancement generation RAG method and AI large language model to achieve real-time voice question and answer interaction, enabling real-time and natural voice interaction with patients during treatment and daily follow-up, providing patient expert-level medical question and answer and psychological counseling services; efficacy feedback evaluation data entry function, used to record the patient's edema status data before daily treatment, including the circumference measurement value of the representative edema site, photos of the marked site, patient self-evaluation score (VAS score, etc.) and doctor score, forming human feedback data for reinforcement learning training;

[0020] The cloud server data center includes a database of expert techniques for treating edema and a generative pre-trained large model of expert techniques for treating edema. The expert technique database stores encrypted multimodal expert treatment training data uploaded from distributed robot terminals. The generative pre-trained large model of expert techniques adopts a Transformer encoder-decoder structure. After collecting a certain amount of expert technique treatment data, it undergoes data training and learning evolution to establish a mapping model from multimodal data input to expert-level technique treatment output. When the system responds to a treatment plan generation request, the processing module performs the following functions: 1) Completely retain the uploaded training data of the treating physicians. 1) Based on the included treatment path spatial location sequence and end-effector posture sequence; 2) Based on the pre-trained large model or any expert treatment data in the automatically matched expert training technique database that matches the current patient's edema state, generate an expert-level force control sequence, movement speed, and rhythm parameters that match the path and posture sequence; 3) Integrate the generated expert-level technique parameters with the current treating physician's original path and posture sequence to form a new individualized, expert-level technique treatment data, which is downloaded to the main control computer via a smart terminal to control the robot to execute; The cloud server data center integrates the Reinforcement Learning from Human Feedback (RLHF) method to optimize the function of the expert technique model, uses the efficacy feedback data uploaded by the smart terminal to construct a reward model for the technique treatment data, and fine-tunes and evolves the pre-trained expert technique model to make the generated technique treatment data more in line with the principle of optimal clinical efficacy.

[0021] The method for implementing the robot system includes the following steps:

[0022] S1. Treatment Preparation and Path Planning: The patient lies flat on the bed with their front or back facing out, and their head is placed in the center of a fixed, non-deformable pillow on the bed. A conical positioning hole that adapts to the head shape is set in the center. Based on the patient's edema area, lymph node distribution, and clinical information, the doctor uses a marking tool to plan the treatment path on the body surface and marks the operation sequence. The robot moves to a suitable treatment position on the side of the bed, ensuring that its operating range covers the treatment area, and the suction cup is fixed to the ground.

[0023] S2. Manual Therapy Training Data Acquisition and Storage: Manual therapy training can be conducted by treating physicians, specialist treating physicians, or patients and their families. The treating physician wears a training glove, achieving synchronized human-machine interaction and manipulation techniques. Pressing the switch button on the training glove activates data communication control, and pressing the training button on the glove initiates the simultaneous acquisition and storage of multimodal data. The physician guides the robotic arm through the marked edema treatment path on the patient's body surface, performing lymphatic drainage techniques. Each treatment path can be trained 1-3 times. Upon completion, the data is saved to the main control computer's memory. Deletion and retraining of each data entry are supported until all treatment paths have been trained and stored. This multimodal data includes: the robotic arm's manipulation operation path position sequence, posture sequence, six-dimensional force applied by the palm, angles of each finger joint, vibration frequency, palm temperature, and short videos of manual therapy recorded by the 3D camera looking down at the bed and the robot's 3D camera.

[0024] S3. Manual therapy training data upload: Users upload the collected manual therapy training data to the cloud server data center through the smart terminal application software. The data is stored in the manual therapy databases of doctors, medical experts, and patients and their families according to the user's qualification certification. If the current user is a treating doctor or a treating medical expert, they can set the repetition number, intensity, speed, and rhythm parameters of each path and the overall path through the smart terminal application software to control the robot to clone and reproduce their training manual therapy process.

[0025] S4. Cloud server generates expert-level manipulation therapy data: The terminal software selects an expert-level manipulation therapy data generation request. After receiving the request, the cloud server data center calls the training data uploaded this time, retains the training treatment path and posture sequence, and uses the expert manipulation pre-trained large model or expert manipulation database to generate new treatment data containing expert-level force, speed and rhythm parameters. The data is downloaded to the robot's main control computer through the smart terminal application software. This method is especially suitable for junior and intermediate edema treatment doctors, interns, patients and their families, and improves the quality of treatment data.

[0026] S5. Automatic Treatment: The number of repetitions, intensity, speed, and rhythm treatment parameters of each path and the overall path are set through the intelligent terminal application software. The robot automatically treats the patient. The robot arm moves according to the path and posture sequence in the training data. The force of the touch on the edematous surface is fed back in real time by a six-dimensional force sensor in a closed loop. It tracks the force sequence set by the expert or the set force to form a force-position hybrid control.

[0027] S6: Adaptive Follow Treatment Steps: During the treatment process, the patient's body position is monitored in real time by the overhead 3D camera, and the original data treatment path coordinates are calculated and adjusted in real time based on visual feedback to achieve adaptive follow.

[0028] S7. Efficacy data feedback and human-computer interaction: Before each day's treatment, patients and doctors can enter data such as edema circumference, photos, and scores through the efficacy feedback interface; during hospitalization and daily home treatment, patients can conduct medical consultations through the AI ​​voice interaction module on the smart terminal;

[0029] S8. Evolution of the Expert Technique Pre-trained Model: The cloud server data center regularly checks the amount of new data on doctor expert technique treatments. Once a threshold is reached, a new round of end-to-end training is initiated. The newly added efficacy feedback data is used for reinforcement learning training with human feedback, and the weights of the pre-trained large model are updated and iterated to evolve.

[0030] (III) Beneficial Effects

[0031] Compared with the prior art, the present invention has the following beneficial technical effects:

[0032] By learning expert techniques for treating tumor edema through end-to-end learning, the robot can precisely reproduce and execute expert-level techniques for treating tumor edema. Ordinary edema specialists only need 1-3 minutes of basic training along the treatment path. The robot then reproduces and executes each 60-minute session of highly repetitive, gentle lymphatic drainage treatment, as required by international standards for edema treatment. This saves doctors dozens of times more treatment time and energy, improving the consistency and standardization of digital treatment. It features visual feedback and path adaptation capabilities, allowing real-time adjustments to treatment coordinates as the patient's position changes, ensuring treatment continuity and safety. It supports remote management of multiple devices by a single doctor, significantly improving treatment efficiency and alleviating the shortage of professional therapists. Digital storage and cloud optimization mechanisms ensure treatment consistency and traceability, facilitating the standardization and promotion of treatment plans. With multi-modal outputs such as vibration and heating, it enhances the patient experience and compliance, making it suitable for various scenarios in hospitals and homes. Attached Figure Description

[0033] Figure 1 : Schematic diagram of the overall structure of the system of this invention;

[0034] Figure 2 : Schematic diagram of the humanoid robotic hand with fitting training gloves of the present invention;

[0035] Figure 3 : Flowchart of the working method of the system of this invention;

[0036] Figure 4 : A schematic diagram of the multimodal data acquisition, processing and cloud data interaction of the system of the present invention.

[0037] Figure label description: top view bed 3D camera (1), bed (2), humanoid robotic hand (3), training glove (4), humanoid robotic arm (5), robot 3D camera (6), main control computer (7), mobile chassis (8), intelligent terminal and application software (9), cloud server data center (10). Detailed Implementation

[0038] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments, which are intended to illustrate the implementation process of the present invention rather than limit the scope of protection of the present invention.

[0039] Example: Treatment of lower extremity lymphedema using the system and method of the present invention.

[0040] Using a patient who developed unilateral left lower extremity lymphedema after gynecological tumor surgery as an example, the implementation process of this system is explained in detail.

[0041] 1. System preparation and patient positioning settings

[0042] First, start the AI ​​robot system described in this invention, move the chassis (8) to a suitable position on one side of the bed (2), and start the electric negative pressure suction cup at the bottom of the chassis (7) to firmly fix the system to the ground and prevent displacement during treatment.

[0043] The patient lies supine on the bed (2), fully relaxed, with the left lower limb slightly externally rotated to fully expose the surface of the inner limb, facilitating treatment. The top-view 3D camera (1) of the system is located in the center above the bed, and its field of view covers the patient's bed, the patient, and the robot's working area beside the bed. After the system is started, the main control computer controls the camera to start collecting initial three-dimensional point cloud data of the patient's body surface, establishes the reference coordinate system {B} of the treatment environment, and detects and identifies key anatomical landmarks such as the anterior superior iliac spine, patella, medial malleolus, and lateral malleolus of the left lower limb as feature points for subsequent position tracking.

[0044] 2. Treatment pathway planning and marking

[0045] The treating physician (hereinafter referred to as "the physician") plans a treatment pathway on the patient based on clinical assessment results such as the stage of lymphedema and lymphangiography images. Based on the principles of the MLD standard procedure, the treatment pathway and sequence in this embodiment are as follows:

[0046] 1) Abdominal lymph node drainage pathway planning and marking: Mark the manual therapy pathway in the patient's left lower abdomen;

[0047] 2) Proximal limb drainage path planning and marking: Starting from the groin area, draw multiple parallel paths along the inner thigh, pointing towards the groin;

[0048] 3) Distal limb drainage path planning and marking: Starting from the ankle, draw multiple parallel paths along the inner side of the lower leg, pointing towards the knee joint;

[0049] 4) The doctor uses a skin safety marking pen to clearly mark the planned treatment path on the patient's left lower limb and abdomen, including the path length, direction, and sequence number.

[0050] 3. Training Mode: Manual Therapy Training Data Acquisition and Storage

[0051] The doctor wears the fitted training gloves (4), presses the glove switch button to connect to power and communication, and the built-in bending sensor transmits the doctor's finger flexion and extension angle data wirelessly to the main control computer (7) in real time. The doctor holds the humanoid robotic hand (3) and presses the "training" button on the back of the glove with his thumb, and the system enters the training mode.

[0052] The doctor strictly follows the path and sequence marked on the body surface, and personally trains the humanoid robotic hand (3) and multi-degree-of-freedom humanoid robotic arm (5) to perform standard MLD techniques (such as gentle static circular, pump-press, etc.). During this process, the system performs end-to-end multimodal data synchronous acquisition, with the two 3D cameras sampling frame rate at 15-30Hz, recording short videos of 5-30s according to the operation duration of each treatment path, and the sampling frequency of other treatment data can be set to 100-500Hz. In this embodiment, the above parameters are set to 30Hz, 15s, and 300Hz respectively, and the data packet is shown in the following formula:

[0053] D t ={t,P t Q t ,F t ,θ t V t ,T t ,I t top ,I t arm}

[0054] in:

[0055] 1) t: timestamp (time in milliseconds).

[0056] 2)P t =(x t ,y t ,z t ): The real-time three-dimensional position coordinates of the center of the palm of the humanoid robot hand (3) under the reference coordinate system {B} (unit: mm).

[0057] 3)Q t =(q t w ,qt x ,q t y ,q t z ): Real-time posture quaternion of the end of the humanoid robotic hand (3).

[0058] 4)F t =(F t x ,F t y ,F t z M t x M t y M t z ): Real-time force / torque data (unit: N, N·m) measured by a six-dimensional force sensor, representing the force and torque in opposite directions of the three-dimensional coordinates. The system monitors the normal force Ftz in real time, combined with z t Force and position are controlled together to ensure that the force is kept stable within the treatment force range of 20-30 mmHg.

[0059] 5)θ t =(θ t1 ,θ t2 ,...,θ tn (4) Training gloves (4) collected the sequence of bending angles of each joint of the doctor's fingers (unit: °).

[0060] 6)V t ,T t The vibration frequency of the vibration module is related to the temperature of the hand heating element.

[0061] 7)I t top ,I t arm RGB-D short video image frames simultaneously captured by the 3D camera (1) on the bed and the 3D camera (6) on the robotic arm.

[0062] After each path training is completed, the doctor can use the "Delete" button on the glove to undo unsatisfactory data segments and retrain. After all paths are trained, the data is encrypted and stored in the local memory of the main control computer (7). After the doctor confirms the data through the smart terminal and application software (9), the anonymized multimodal training dataset D collected this time is sent to the computer. t Uploaded to the cloud server data center (10), stored in the doctor's manual therapy database. If the doctor is a registered and certified chief physician, the manual therapy database is stored in the expert manual therapy database for training and optimization of the expert manual therapy large model.

[0063] 4. Generate and download expert-level manual therapy data.

[0064] The cloud server data center (10) receives and stores the doctor's treatment data uploaded this time. When the system needs to generate expert-level treatment data for the patient, the cloud processing module performs the following steps:

[0065] 1) Treatment path and posture data preservation: Preserve the spatial position sequence P representing the treatment path of the robotic arm from the current doctor training data. t and robotic arm pose sequence Q t This ensures that treatment data follows the treatment pathway that the doctor trained for the patient.

[0066] 2) Generating optimized manipulation parameters: Based on the pre-trained large model of the expert manipulation techniques or by selecting high-quality manipulation data from any expert in the expert manipulation technique database, generate an expert-level force control sequence F that matches the path and posture sequence. t Speed ​​and rhythm parameters.

[0067] 3) Fusion generation and download: The expert-level manipulation parameters generated above are fused with the treatment path and posture sequence of the robot hand trained by the doctor to generate a new expert-level, personalized digital treatment data. This data is downloaded to the robot's main control computer (7) through the smart terminal application software (8). After setting the treatment parameters (such as the number of treatment cycles: 10 times, total treatment time: 60 minutes) through the application software, click the "Treatment" button to start automatic execution. The multi-degree-of-freedom humanoid robot arm (5) and the humanoid robot hand (3) start to automatically execute the new treatment data.

[0068] 5. Treatment Implementation, Interaction, and Feedback

[0069] During the treatment, the robotic arm's movement trajectory is determined by P in the treatment data. t Sequential control, with the end contact force controlled in real-time by a six-dimensional force sensor using closed-loop feedback, tracks the F value in the treatment data. t The sequence ensures that the intensity is precisely and stably maintained within the treatment window of 20-30 mmHg.

[0070] During treatment, patients can use the AI ​​voice interaction module built into the smart terminal application software (8) to communicate in real time with the large language model and the professional knowledge base of tumor rehabilitation and psychology. For example, if a patient asks, "I feel a little sore and swollen during treatment today, is that normal?", the system retrieves information from the medical knowledge base and psychology through speech recognition and reference retrieval augmented generation (RAG) technology, and generates the following voice reply through logical reasoning of the large language model: "Hello, it is normal to feel a slight soreness and swollenness during treatment, which indicates that lymph fluid is flowing. Please relax, and tell me immediately if you experience any stinging." The system provides professional and patient real-time answers, relieving the patient's anxiety.

[0071] During treatment, when the patient slightly moved their left leg, the 3D camera (1) detected the displacement Δd of the feature points on the body surface in real time. The visual recognition algorithm in the main control computer (7) calculated the coordinate transformation matrix of the marked treatment path and performed real-time remapping calculation on the treatment path.

[0072] P t '=R·P t +Tr

[0073] Where R and Tr are the rotation matrix and translation vector calculated based on the changes in feature points, respectively, and P t P is the stored coordinate of the treatment path planning marked on the body surface on the day. t The updated coordinates enable the robotic arm to adaptively follow the patient's position markings and treatment path, ensuring the continuity and safety of the treatment operation and avoiding the hassle of interrupting treatment and repositioning due to patient movement.

[0074] 6. Treatment feedback and RLHF model optimization

[0075] After this and subsequent treatments, both the patient and the doctor continuously entered data such as subjective feelings scores, circumference measurements and photos of representative edema sites, and doctor's comments through the efficacy feedback interface of the smart terminal and application software (9), forming a series of high-quality human feedback datasets. These anonymized data were uploaded to the cloud server data center (10) for storage after obtaining informed consent.

[0076] After accumulating sufficient high-quality manipulation data and corresponding human feedback data, the cloud server initiates a reinforcement learning (RLHF) training process based on human feedback. It uses the feedback data to build a reward model and fine-tunes and optimizes the pre-trained large model of the treatment manipulation, so that the model is more inclined to generate treatment strategies with better clinical efficacy in future treatment plan generation, thereby achieving continuous iteration of the treatment model.

[0077] 7. Treatment efficacy

[0078] This embodiment demonstrates that the system of the present invention can achieve the following effects:

[0079] 1) Generation of expert-level, personalized manual therapy data: By integrating current doctor training data according to treatment pathways with cloud-based expert-level manual parameters, high-quality digital treatment data at the level of doctor's expert manual techniques is generated.

[0080] 2) Adaptive Intelligence and Interactive Experience: The system uses machine vision to perceive changes in the patient's body position in real time and then adaptively follows the treatment. Based on the tumor rehabilitation knowledge base and large language model, it enables real-time voice question-and-answer interaction between AI experts and patients, providing expert-level psychological communication and significantly improving treatment safety and patient compliance.

[0081] 3) Data-driven continuous optimization: By collecting a large amount of expert techniques and efficacy feedback data for treating edema, [the following is adopted]...

[0082] The RLHF mechanism optimizes expert technique models, enabling them to continuously approach best clinical practice, thus solving the problem of needing batch training and continuous evolution of models.

[0083] 4) High efficiency and resource optimization: It greatly reduces the workload of doctors and realizes the replication and efficient use of multi-expert data experience through centralized cloud management and optimization of treatment plans.

[0084] In summary, this invention achieves robot learning and reproduction of expert-level treatment techniques through a closed loop of treatment technique data collection, cloud-based fusion generation, clinical feedback, and model iteration. It addresses the pain points in lymphedema treatment, such as fatigue, inefficiency, scarcity of experts, difficulty in standardization, and difficulty for patients to perform self-treatment at home. It provides a practical and continuously evolving digital and intelligent solution.

Claims

1. An AI robotic system for end-to-end learning expert techniques to treat tumor edema, characterized in that, include: Overhead bed 3D camera, humanoid robotic hand with fitted training gloves, multi-degree-of-freedom humanoid robotic arm, robot 3D camera, main control computer, mobile chassis, intelligent terminal and application software and cloud server data center. The overhead 3D camera is positioned above the bed to collect three-dimensional image data of the patient's body and identify changes in body position in real time. The humanoid robotic hand with fitted training gloves has thin-film dot matrix tactile sensors arranged on the surface of its palm and fingers, and integrates a vibration motor and heating module inside; the training gloves have built-in sensors for the bending angle of each finger joint to capture the finger movements and postures of the trainee, and control buttons are provided on its back. The multi-degree-of-freedom humanoid robotic arm has its end cap connected to the humanoid robotic hand via a six-dimensional force sensor. The robot's 3D camera is mounted on the horizontal rotating joint on the upper part of the robot arm base, and is used to record the manual therapy process video from a horizontal perspective and for safety monitoring. The main control computer is used to integrate multimodal data acquisition, storage, communication and motion control, and to execute new treatment data from the cloud; The mobile chassis is used to provide the system with mobility, ground fixation, and backup power supply; The intelligent terminal and application software are used to provide a human-computer interaction interface, treatment parameter settings, treatment process monitoring, and integrate an AI voice interaction module and efficacy feedback evaluation data input function. The cloud server data center includes a large model of expert techniques for treating edema and an expert technique database. It automatically stores multimodal data of technique training and trains and iterates the large model of expert techniques as it accumulates the required amount of expert technique data and feedback efficacy evaluation data.

2. The system according to claim 1, characterized in that, The AI ​​voice interaction module integrated into the smart terminal and application software includes a rehabilitation professional knowledge base and a patient psychology knowledge base. It uses the retrieval enhancement generation RAG method and AI large language model to realize real-time voice interaction with patients and provide medical Q&A and psychological counseling services.

3. The system according to claim 1, characterized in that, The intelligent terminal and application software integrate therapeutic feedback assessment data entry functions, which are used to record the circumference measurement value of the representative edema site of the patient, the photo of the marked site, the patient's self-perception score and the doctor's comments, forming consistent human feedback data and uploading it to the cloud server data center.

4. The system according to claim 1, characterized in that, The cloud server data center, once it accumulates enough expert manipulation data to meet the quantity requirements, trains a pre-trained large model of expert manipulation techniques, and then optimizes the large model of expert manipulation techniques by combining reinforcement learning training with human feedback efficacy evaluation data.

5. The system according to claim 1, characterized in that, In automatic treatment mode, the main control computer controls the robotic arm to execute new treatment data generated by the cloud. The force of the robotic arm contacting the body surface is fed back in real time by a six-dimensional force sensor to control the force in the treatment data.

6. The system according to claim 1, characterized in that, The 3D camera at the top of the bed identifies feature points on the patient's body surface in real time. When the patient's body position is detected to move, the visual detection algorithm in the main control computer calculates the coordinate transformation matrix of the treatment path as the body position moves, and performs real-time remapping of the pre-stored planned treatment path, so as to realize the robot arm's adaptive following treatment of the patient's treatment path.

7. A method for treating tumor edema using an end-to-end learning expert technique based on the system described in any one of claims 1-6, characterized in that, Includes the following steps: S1. Treatment preparation and path planning: The treating physician plans and marks the treatment path on the patient's body surface according to the area of ​​edema, and sets the robot at the bedside to a suitable treatment position; S2. Manual therapy training data collection and storage: The user wears training gloves and guides the humanoid robotic arm and humanoid robotic hand to perform manual operations along the marked path. The system synchronously collects multimodal training data. Users include treating physicians, treating physician experts, patients and their families. When the current user is a treating physician or a treating physician expert, the number of repetitions, intensity, speed and rhythm parameters of each path and the overall path can be set through the intelligent terminal application software, and the robot can be directly controlled to automatically reproduce the trained manual therapy. S3. Upload of manual therapy training data: Collect user multimodal manual therapy training data and upload it to the cloud server data center, and store it in the manual therapy training database corresponding to the user type; S4. Cloud server generates expert-level manual therapy data: The application software controls the cloud server data center to retrieve the currently uploaded training data, selects expert data or expert manual large model matching the expert manual database, generates expert-level force, speed and rhythm parameters, integrates the treatment path and posture sequence of the robotic hand in the training data, generates new expert-level force control treatment data, and downloads the solution to the main control computer. S5. Automatic Treatment: Treatment parameters are set by the intelligent terminal application software, and the main control computer controls the robot to automatically execute expert-level treatment data; S6. Adaptive following: During the treatment process, the patient's position is monitored in real time by the 3D camera on the overhead bed, and the treatment coordinates are adjusted in real time according to visual feedback to achieve adaptive following; S7. Collection of therapeutic feedback data: Collect therapeutic feedback evaluation data through the smart terminal and application software and upload it to the cloud server data center; S8. Iterative optimization of the expert manipulation model: The cloud server data center periodically uses newly added expert manipulation treatment data and newly added efficacy feedback data to iteratively train and optimize the expert manipulation model.

8. The method according to claim 7, characterized in that, The multimodal data acquisition function described in step S3 acquires data packets in the form of D. t Represented as: D t ={t,P t ,Q t ,F t ,θ t ,V t ,T t ,I t top ,I t arm } Where t is the timestamp, P t Q represents the three-dimensional coordinates of the center of the humanoid robotic hand moving along the treatment path. t For the humanoid robotic hand posture quaternion, F t θ represents real-time force / torque data measured by a six-dimensional force sensor. t The sequence of flexion angles of each joint of the doctor's fingers was collected for training gloves. t T is the intensity frequency of the vibration module. t For the temperature of the heating module, I t opt It is a short video of the manual therapy process captured by a 3D camera looking down at the bed. t arm It is a short video of the manual therapy process captured by the 3D camera of the arm-mounted robot.

Images