A physiotherapy area executable computing display method, system and device

Abstract

The application discloses a physiotherapy area executable calculation display method, system and device, and belongs to the technical field of physiotherapy device control. S1, an RGB-D image of a human body is collected through a depth camera to obtain a human body contour point set H_base; S2, H_base is converted to a robot base coordinate system through hand-eye calibration to form a robot workable area R_base; S3, set intersection of H_base and R_base is obtained based on voxel hashing to obtain a physiotherapy executable intersection area I; S4, I is mapped to a screen coordinate through perspective projection and is displayed in a semi-transparent overlay form on a host computer interface; and S5, a safety mode state variable S is set, and the operator candidate points are respectively constrained in a neighborhood of I or R_base according to the S state, and an operator who does not meet the condition is rejected and prompted. The application realizes real-time coupling calculation and visual feedback of the robot reachable space and the human body area, and effectively improves the physiotherapy operation planning efficiency and safety.

Description

Technical Field

[0001] This invention belongs to the field of physiotherapy equipment control technology, specifically a physiotherapy area executable calculation and display method, system and device. Background Technology

[0002] In the fields of physical therapy, traditional Chinese massage, and sports rehabilitation, precise positioning of surgical points is a core prerequisite for ensuring efficacy and safety. In recent years, with the rapid development of artificial intelligence, machine vision, and robot control technology, physical therapy robots, as an important branch of medical robots, are gradually moving from the laboratory to clinical applications. Medical robots are clearly defined as robots used in the medical field to perform mechanical work or assist medical activities, and are mainly divided into four categories: surgical robots, rehabilitation robots, diagnostic robots, and other medical robots. As an important component of rehabilitation robots, physical therapy robots have shown great potential in alleviating the workload of therapists and improving the accessibility of physical therapy services.

[0003] In traditional clinical procedures, operators use a 3D human-computer interaction interface to plan treatment trajectories or specify manipulation points for the robot, which then performs therapeutic operations such as massage, acupressure, laser therapy, and electrical stimulation on the patient's body surface. This human-computer collaborative process has been gradually adopted, but several technical bottlenecks remain in terms of automated execution: First, the reachable space of the robot's end effector head is jointly constrained by the robot's mechanical structure, joint physical limitations, and the feasible region of inverse kinematics, meaning there is a clearly defined workspace boundary. If the manipulation points set by the operator in the 3D interactive interface are outside this boundary, the system will experience inverse kinematics failure or joint over-limit anomalies during execution, making it unable to complete the planned trajectory. Existing solutions typically report such errors only during execution, forcing the operation to be interrupted and increasing replanning overhead, severely impacting clinical operation efficiency. In the field of rehabilitation robotics, inverse kinematics solving is itself a complex nonlinear problem with a large computational load, and traditional numerical iterative methods are particularly inefficient when dealing with batch judgment of multiple candidate points.

[0004] Secondly, from a safety perspective, physiotherapy procedures require that the treatment points strictly fall within the target human body area to avoid damage to adjacent non-target tissues or causing external impaction. However, existing systems lack a closed-loop mechanism for real-time spatial comparison, intersection calculation, and visualization of constraints between the robot's workable area and the human anatomical area. Operators can only rely on subjective experience to judge the legitimacy of the treatment points, posing significant safety risks. Although some studies have explored human-robot collaborative safety space visualization methods based on mixed reality, a systematic solution for jointly modeling robot kinematic reachability constraints and human body region attribution constraints has not yet been found.

[0005] Third, existing user interfaces cannot intuitively present an effective physiotherapy execution area that simultaneously satisfies robot accessibility and human body area attribution, resulting in low planning efficiency and a high probability of misoperation. When faced with complex multi-point physiotherapy plans, the time spent by operators repeatedly trying and adjusting the points is significant, severely hindering the clinical promotion of physiotherapy robots. 3D visualization and human-computer interaction have been identified as key technological directions for medical robots, but in terms of visualizing the working area of ​​physiotherapy robots, existing research still focuses on static workspace analysis, lacking dynamic, real-time interactive visualization solutions tailored to clinical operation scenarios.

[0006] Fourth, there is a lack of universal solutions and dynamic safety constraint mechanisms across different body parts. Existing computer-aided planning schemes are mostly designed for single body surface areas or fixed postures, making it difficult to adapt to the differentiated morphological characteristics of multiple body parts such as the neck, shoulders, back, waist, and limbs, as well as the safety level requirements of different clinical scenarios. Their versatility and flexibility are significantly insufficient. Furthermore, the safety mechanisms of existing physiotherapy robots mainly focus on force control and emergency stop protection during operation, lacking state variable control and dynamic constraint mechanisms based on safety modes during the operation planning stage.

[0007] In recent years, YOLO v9, as the latest representative architecture in the field of object detection, has demonstrated superior performance compared to previous models in human instance segmentation under complex backgrounds. Its core innovations include the Programmable Gradient Information (PGI) mechanism and the Generalized Efficient Layer Aggregation Network (Gelan-C), achieving inference speeds exceeding 60fps and providing the technical foundation to meet the needs of real-time physiotherapy navigation. However, there are currently no patents or technical reports on the joint modeling of human region perception and robot kinematic feasibility constraints, nor on the provision of dynamic safety mode constraint mechanisms. This invention addresses this technological gap by proposing a complete closed-loop technical solution from human recognition to safe robot interaction. Summary of the Invention

[0008] To solve the above problems, the technical solution provided by the present invention is as follows: The present invention provides a method for performing calculations and displaying data in a physiotherapy area, comprising the following steps: S1. Human contour recognition stage: The human body RGB image and depth map are acquired by the depth camera. The human body mask is output by the YOLOv9 instance segmentation network. The coordinate back projection of the pixels in the mask is performed according to the depth camera intrinsic parameter matrix K and the depth value D(u,v). After statistical outlier removal filtering, the human contour point set H_base is obtained. S2, Robot workable area calculation stage: The transformation matrix T_cam→base∈SE(3) obtained by hand-eye calibration is used to transform the camera coordinate system point set to the robot base coordinate system. For each point in the candidate point set after voxel sampling, the target pose with surface normal vector alignment constraint is constructed. The inverse kinematics is solved by Levenberg-Marquardt numerical iteration, and the joint limit constraint and Jacobian matrix condition number constraint κ(J(q))<κ_max are verified at the same time. The points that pass all constraints are marked as reachable points, forming the robot workable area R_base; S3, Intersection Region Calculation Stage: Based on the voxel hash table, perform set intersection calculation on H_base and R_base in O(N) time complexity to obtain the physiotherapy executable intersection region I=H_base∩R_base; S4, Human-computer interaction interface feedback stage: By observing the matrix V_view and the perspective projection matrix P_proj, each point in I is mapped to screen coordinates and rendered in the form of a high-brightness cloud on the 3D human-computer interaction interface. S5, Safety Mode Switch Constraint Phase: Set the safety mode state variable S∈{0,1}. When S=1, the operator's selected method candidate point p_op must satisfy min{‖p−p_op‖:p∈I}≤d_thr; when S=0, it must satisfy min{‖p−p_op‖:p∈R_base}≤d_thr; if the conditions are not met, the operation will be rejected and a prompt will be given.

[0009] Preferably, step S1 includes S11. Image Acquisition and Preprocessing: The depth camera acquires color images I_rgb∈R^(H×W×3) and depth maps D∈R^(H×W) at 30fps. CLAHE contrast equalization is performed on I_rgb; 3×3 median filtering is performed on D to eliminate depth noise. S12, Instance Segmentation: Input the preprocessed I_rgb into the YOLOv9 instance segmentation network (pre-trained with COCO, fine-tuned with a special dataset), outputting a human instance mask M∈{0,1}^(H×W), with a confidence threshold τ_conf=0.75. After performing morphological closing operations on the mask, downsampling is performed to obtain the set of boundary pixels: U={(u,v)|M(u,v)=1}; S13, Coordinate Backprojection: Using the depth camera intrinsic parameter matrix K and the depth map, each pixel (u,v) in U is backprojected to the camera coordinate system according to the pinhole camera model: [xc,yc,zc]ᵀ=D(u,v)·K⁻¹·[u,v,1]ᵀ Where K = [[fx,0,cx],[0,fy,cy],[0,0,1]]; S14. Validity Filtering and Outlier Removal: Invalid points with D(u,v)=0 or D(u,v)>1.5m are removed. Statistical Outlier Removal (SOR) is applied to the remaining point set with the number of neighborhood points k_sor=30 and the standard deviation factor λ_sor=2.0. Finally, the human point set H_cam in the camera coordinate system is obtained.

[0010] Preferably, step S2 includes S21, Coordinate System Transformation: Obtain the homogeneous transformation matrix T_cam→base∈SE(3) from the camera coordinate system to the robot base coordinate system through hand-eye calibration, and transform H_cam to the robot base coordinate system: [xb,yb,zb,1]ᵀ=T_cam→base·[xc,yc,zc,1]ᵀ The resulting point set is denoted as H_base. Using voxel side length δ=5mm for uniform sampling, and taking the centroid of each voxel, a candidate point set P={p1,p2,…,pN} is obtained. S22, Target Pose Construction: For each point p=(xb,yb,zb)ᵀ in P, estimate the human body surface normal vector n∈R³ through local point cloud PCA, and determine the tool head target pose R_tool∈SO(3), requiring the end z-axis to be aligned with the normal vector: arccos(nᵀ·R_tool·ez)<θ_max,θ_max=5°; S23, LM Inverse Kinematics Solution: The objective is to find a joint angle q such that FK(q) = T_target, while simultaneously satisfying the joint constraint qi_min ≤ qi ≤ qi_max, the singularity avoidance constraint κ(J(q)) < 100, and the convergence accuracy constraints ‖Δp‖ < 1 mm, ‖Δφ‖ < 0.5°. The joint update amount in the k-th iteration is: Δq⁽ᵏ⁾=−(JᵀJ+λ⁽ᵏ⁾I)⁻¹·Jᵀ·e⁽ᵏ⁾ Where e⁽ᵏ⁾∈R 6 Let λ be the end pose error vector, λ⁽ᵏ⁾ be the adaptive damping coefficient (initially 0.01, adjustment factor β=10), k_max=100, and add it to R_base if all constraints are satisfied, otherwise discard it.

[0011] Preferably, step S3 specifically involves: In discrete point set representation, the intersection region for physical therapy is defined as: I=H_base∩R_base={p∈P|p∈H_base∧p∈R_base} Spatial hashing is used to accelerate intersection calculation. The time complexity of set member testing is O(1), and the total time complexity of intersection of the entire set is O(N). The voxel hash function is defined as: key(p,δ)=⌊px / δ⌋·P1⊕⌊py / δ⌋·P2⊕⌊pz / δ⌋·P3 Where P1=73856093, P2=19349663, P3=83492791 are preset large prime numbers, ⊕ represents bitwise XOR, and ⌊·⌋ represents rounding down.

[0012] Preferably, step S4 includes S41, Perspective Projection: Each 3D point in the intersection point set I is mapped to the homogeneous clipping space after transformation by the view matrix V_view and the perspective projection matrix P_proj. [uc,vc,w]ᵀ=P_proj·V_view·[xb,yb,zb,1]ᵀ The screen pixel coordinates obtained by perspective division and viewport transformation are: xs = (uc / w + 1)·W / 2, ys = (vc / w + 1)·H / 2; S42. Mask Rendering and Host Computer Display: The system overlays the calculated intersection region I and the non-workable region onto the color image of the depth camera in the form of a semi-transparent mask, and displays it in real time through the host computer APP interface. The workable region (I) is highlighted with a green mask (RGB(0,255,0), α=0.75), and the non-workable region is covered with a red semi-transparent mask, so that the operator can intuitively identify the workable and non-workable areas on the patient's body surface. The screen radius of each point in the mask is adaptive according to the observation depth: r_screen = r_world·fy_proj / z_view, where r_world=δ / 2.

[0013] Preferably, step S5 includes S51. Operation point acquisition: The operator sets candidate points on the 3D interface using the mouse, and the system obtains the candidate points p_op∈R³ through ray casting; S52. Safety Mode Criterion: The Boolean variable S∈{0,1} is bound to the interface checkbox, and the system applies real-time constraints based on the following criteria: When S=1: min{‖p − p_op‖ : p ∈ I} ≤ d_thr = 10mm; When S=0: min{‖p − p_op‖ : p ∈ R_base} ≤ d_thr = 10mm; Nearest neighbor search is implemented using a KD-Tree, with a single query time complexity of O(log N). When the conditions are met, p_op is added to the manipulation point list and displayed in blue; when the conditions are not met, a prompt dialog box pops up to reject the operation. S=1 corresponds to strict double constraints (formal clinical physiotherapy), and S=0 corresponds to a single reachability constraint (offline debugging).

[0014] A physiotherapy area computational display system for implementing the above method includes: The depth camera module is used to acquire RGB images and depth information of the human body; The visual processing module loads the YOLO v9 instance segmentation model and performs point cloud filtering and coordinate back projection. The robot control module stores DH parameters and joint limits, and performs inverse LM calculations and reachability determination. The intersection calculation module calculates H_base∩R_base using voxel hashing; the human-computer interaction module performs perspective projection rendering and KD-Tree nearest neighbor constraint determination. The safety mode switch is bound to a Boolean variable S∈{0,1} and controls the range of point constraints.

[0015] A physiotherapy robot includes a robotic arm, an end-effector, an RGB-D camera, a controller, and a memory. The controller is used to execute the above-described method, calculate and display the intersection of the robot's working area and the human body area in real time, and dynamically constrain the operator's position settings according to the safety mode switch status.

[0016] Beneficial effects Compared with the prior art, the technical solution provided by this invention has the following advantages: This invention provides a method, system, and device for performing computational display in a physiotherapy area. The method includes: S1, acquiring RGB images and depth maps of the human body using a depth camera, outputting a human body mask using a YOLOv9 instance segmentation network and performing coordinate back projection, and obtaining a human body contour point set H_base through statistical outlier removal filtering; S2, transforming H_base to the robot base coordinate system through hand-eye calibration, constructing a target pose with normal vector alignment constraints on the candidate point set after voxelization sampling, solving the inverse kinematics using the Levenberg-Marquardt algorithm, and filtering reachable points by combining joint constraints and Jacobian condition number constraints. S3. Based on voxel hashing, perform O(N) time complexity set intersection on H_base and R_base to obtain the physiotherapy executable intersection region I; S4. Map I to screen coordinates through perspective projection and display it on the host computer interface as a semi-transparent overlay, where the workable area is highlighted in green and the inoperable area is covered in red; S5. Set the safety mode state variable S∈{0,1}. When S=1, the operator's selected technique candidate point must fall within 10mm of I, and when S=0, it must fall within 10mm of R_base. Otherwise, the operation is rejected and a prompt is displayed. This invention realizes real-time coupled calculation and visualization feedback between the reachable space of the physiotherapy robot and the human body area, effectively improving the efficiency and safety of operation planning, and is suitable for physiotherapy scenarios of multiple parts such as the neck, shoulders, back, waist, and limbs. Attached Figure Description

[0017] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the three-level coordinate transformation relationship of the present invention; Figure 3 This is a flowchart of the LM inverse kinematics feasibility determination algorithm of the present invention; Figure 4 These are comparison images of the overlay display effect of the host computer APP interface of the present invention; Figure 5 This is a schematic diagram of the system hardware composition and signal flow of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] In this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated device, element, or component to having a specific orientation, or to be constructed and operated in a specific orientation.

[0021] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0022] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linking," and "socketing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral structure; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Example 1 See attached document Figure 1-5This embodiment presents a method for calculating and displaying a physiotherapy area. This method aims to address the problems of low operational efficiency and insufficient safety in physiotherapy robot operation planning caused by the inability to predict the overlap between the robot's reachable space and the human body area in real time. This embodiment forms a complete closed loop from physical space perception to human-machine interaction control through five core stages: human body contour recognition, robot workable area calculation, intersection area calculation, human-machine interface feedback, and safety mode switch constraints. The following provides a detailed description of each step.

[0025] The method in this embodiment includes the following steps: S1. Human Contour Recognition Stage: RGB images and depth maps of the human body are acquired using a depth camera. A human body mask is output using a YOLOv9 instance segmentation network. Pixels within the mask are back-projected onto their coordinates based on the depth camera's intrinsic parameter matrix K and the depth value D(u,v). After statistical outlier removal filtering, the human contour point set H_base is obtained. This step is the data source for all subsequent spatial calculations; its segmentation accuracy and point cloud quality directly determine the reliability of the final intersection region.

[0026] Specifically, S11, Image Acquisition and Preprocessing: The depth camera acquires color images I_rgb∈R^(H×W×3) and depth maps D∈R^(H×W) at 30fps. To improve illumination adaptability, I_rgb is converted to the HSV color space, and its luminance (V) channel is subjected to CLAHE contrast equalization before being converted back to the RGB space to enhance local texture and avoid color distortion. A 3×3 median filter is applied to D to eliminate isolated depth jump points caused by sensor speckle noise.

[0027] S12, Instance Segmentation: The preprocessed I_rgb is input into the YOLOv9 instance segmentation network (pre-trained on COCO, fine-tuned on a specialized dataset). YOLOv9's Programmable Gradient Information (PGI) mechanism preserves the integrity of gradients during forward propagation through an auxiliary invertible branch, effectively alleviating the information bottleneck problem of deep networks. Its Generalized Efficient Layer Aggregation Network (GELAN-C) adopts a lightweight cross-stage fusion architecture, enabling the model to output a high-confidence human instance mask M∈{0,1}^(H×W) even in complex backgrounds (such as patients covered with treatment towels or wearing dark clothing), with an inference speed exceeding 60fps, meeting real-time processing requirements. A confidence threshold τ_conf=0.75 is set to balance segmentation recall and false positive rate, and morphological closing operations are performed on the mask to fill holes. Then, downsampling is performed to obtain the boundary pixel set U={(u,v)|M(u,v)=1}, which reduces the subsequent projection computation by about 60%.

[0028] S13, Coordinate Back Projection: Using the depth camera intrinsic parameter matrix K and the depth map, each pixel (u,v) in U is back projected to the camera coordinate system according to the pinhole camera model: [xc,yc,zc]^T=D(u,v)·K⁻¹·[u,v,1]^T, where K=[[fx,0,cx],[0,fy,cy],[0,0,1]).

[0029] S14. Validity Filtering and Outlier Removal: Invalid points with D(u,v)=0 or D(u,v)>1.5m are removed—the 1.5m distance threshold is set based on the typical working distance of the physiotherapy robot (usually 0.8~1.4m) to preemptively truncate background interference. Statistical Outlier Removal (SOR) is applied to the remaining point set, with the number of neighborhood points k_sor=30 and the standard deviation factor λ_sor=2.0. The principle of this filtering method is based on Chebyshev's inequality: under any distribution, the probability of deviating from the mean by more than λ times the standard deviation does not exceed 1 / λ². When λ=2.0, approximately 5% of outlier noise points can be effectively removed, while fully preserving the subtle curvature features of the human body surface (such as spinal grooves and scapular prominences), ultimately obtaining the human body point set H_cam in the camera coordinate system.

[0030] S2, Robot Workable Region Calculation Stage: The transformation matrix T_cam→base∈SE(3) obtained through hand-eye calibration is used to transform the camera coordinate system point set to the robot base coordinate system. For each point in the candidate point set after voxel sampling, a target pose with surface normal vector alignment constraints is constructed. The inverse kinematics is solved through Levenberg-Marquardt numerical iteration, and the joint limit constraints and Jacobian matrix condition number constraints κ(J(q))<κ_max are verified at the same time. Points that pass all constraints are marked as reachable points, forming the robot's workable region R_base. This step quantizes the abstract robot kinematic feasible region into a discrete point cloud in the base coordinate system, which is a prerequisite for subsequent intersection calculation.

[0031] Specifically, S21, coordinate system transformation: obtain the homogeneous transformation matrix T_cam→base∈SE(3) from the camera coordinate system to the robot base coordinate system through hand-eye calibration, and transform H_cam to the robot base coordinate system: [xb,yb,zb,1]^T=T_cam→base·[xc,yc,zc,1]^T, and the resulting point set is denoted as H_base. Uniform sampling with voxel side length δ=5mm, taking the centroid of each voxel, to obtain the candidate point set P={p1,p2,...,pN}. The selection basis of δ value is: the working diameter of the physiotherapy tool head is usually 20~40mm, δ=5mm corresponds to 1 / 16 to 1 / 4 of the working area, which is sufficient to capture the undulation of the human body surface without producing a step effect, while controlling the point cloud scale to 10 5 Within a certain order of magnitude, it also takes into account the computational burden of subsequent batch inverse LM solutions.

[0032] S22. Target pose construction: For each point p=(xb,yb,zb)^T in P, estimate the human body surface normal vector n∈R³ through local point cloud principal component analysis (PCA), and perform sign consistency reversal on the eigenvector output by PCA—calculate the dot product of this vector with the direction of the camera optical center (the vector pointing from the camera origin to the point cloud). If the dot product is less than 0, flip the direction to ensure that the normal vector uniformly points to the camera (i.e., to the outside of the body surface). Based on this, determine the target pose of the tool head R_tool∈SO(3), requiring the end effector z-axis to be aligned with the normal vector: arccos(n^T·R_tool·e_z)<θ_max,θ_max=5°. This pose constraint ensures that the physiotherapy tool (such as a pressure head or laser probe) applies force perpendicular to the body surface, avoiding patient discomfort or positioning deviation due to tangential sliding.

[0033] S23. LM Inverse Kinematics Solution: The objective is to find a joint angle q such that the forward kinematics FK(q) = T_target, while simultaneously satisfying the joint constraint qi_min ≤ qi ≤ qi_max, the singularity avoidance constraint κ(J(q)) < 100, and the convergence accuracy constraints ‖Δp‖ < 1 mm, ‖Δφ‖ < 0.5°. The joint update amount in the k-th iteration is Δq^(k) = −(J^TJ + λ^(k)I)⁻¹·J^T·e^(k), where e^(k) ∈ R. 6 Let λ be the end-effector pose error vector, λ^(k) be the adaptive damping coefficient (initially 0.01, adjustment factor β=10), and k_max=100. The core principle of this LM algorithm is to combine the local second-order convergence of the Gauss-Newton method with the global search capability of the gradient descent method: when the residual is large, λ automatically increases, and the algorithm tends towards the steepest descent method to ensure convergence stability; when approaching the optimal solution, λ decreases, and the algorithm tends towards the Gauss-Newton method to accelerate convergence. The physical meaning of the condition number constraint κ(J)<100 is to ensure that the mapping from the joint velocity of the robotic arm to the Cartesian velocity of the end effector has a balanced transmission ratio—according to the operability theory, the reciprocal of κ is the operability. This threshold ensures that the robotic arm has sufficient motion flexibility in all directions. Experimental statistics show that this value can reduce the joint acceleration RMS by about 30% during trajectory execution, effectively avoiding speed loss due to approaching singular configurations. If all constraints are satisfied, the point is marked as reachable and added to R_base; otherwise, it is discarded.

[0034] S3. Intersection Region Calculation Stage: Based on the voxel hash table, perform set intersection calculation on H_base and R_base in O(N) time complexity to obtain the therapeutic executable intersection region I=H_base∩R_base. In discrete point set representation, I={p∈P|p∈H_base∧p∈R_base}. Spatial hashing is used to accelerate the intersection calculation, with set member testing time complexity of O(1) and total set intersection time complexity of O(N). The voxel hash function is defined as key(p,δ)=⌊px / δ⌋·P1⊕⌊py / δ⌋·P2⊕⌊pz / δ⌋·P3, where P1=73856093, P2=19349663, P3=83492791 are preset large prime numbers, ⊕ represents bitwise XOR, and ⌊·⌋ is rounding down.

[0035] S4, Human-Computer Interaction Interface Feedback Stage: By observing the matrix V_view and the perspective projection matrix P_proj, each point in I is mapped to screen coordinates and rendered in the form of a high-brightness cloud on the 3D human-computer interaction interface to achieve "what you see is what you get" spatial constraint visualization.

[0036] Specifically, S41, perspective projection: Each 3D point in the intersection point set I is mapped to the homogeneous clipping space after being transformed by the observation matrix V_view and the perspective projection matrix P_proj: [uc,vc,w]^T=P_proj·V_view·[xb,yb,zb,1]^T. After perspective division and viewport transformation, the screen pixel coordinates are obtained: xs=(uc / w+1)·W / 2, ys=(vc / w+1)·H / 2.

[0037] S42. Masking Rendering and Host Computer Display: The system overlays the calculated intersection region I and the non-workable region onto the color image from the depth camera as a semi-transparent mask, and displays it in real time via the host computer APP interface. The workable region (I) is highlighted with a green mask (RGB(0,255,0), α=0.75), while the non-workable region is covered with a red semi-transparent mask. The choice of green and red refers to internationally accepted safety warning semantics, significantly reducing the operator's cognitive recognition time. The screen radius of each point in the mask adapts to the observation depth: r_screen = r_world·fy_proj / z_view, where r_world=δ / 2. This adaptive mechanism is based on the principle of visual constancy—the projected size of distant objects on the screen should be proportionally reduced to ensure that the point cloud particles at different depths are visually uniform and consistent, avoiding overly dense near objects forming color blocks and overly sparse distant objects causing missed recognition.

[0038] S5, Safety Mode Switching Constraint Phase: Set the safety mode state variable S∈{0,1}. When S=1, the operator's chosen method candidate point p_op must satisfy min{‖p−p_op‖:p∈I}≤d_thr; when S=0, it must satisfy min{‖p−p_op‖:p∈R_base}≤d_thr. Operations not meeting these conditions are rejected and a prompt is given. This step is the final execution checkpoint of the system's safety mechanism, transforming all the aforementioned spatial calculation results into operational constraints.

[0039] Specifically, S51, Operation Point Acquisition: The operator sets candidate points on the 3D interface using the mouse, and the system obtains the candidate points p_op∈R³ through ray casting.

[0040] S52. Safety Mode Criteria: The Boolean variable S∈{0,1} is bound to the interface checkbox. The system applies real-time constraints based on the following criteria: When S=1: min{ ‖p − p_op‖ : p ∈ I} ≤ d_thr = 10mm; When S=0: min{ ‖p − p_op‖ : p ∈ R_base} ≤ d_thr = 10mm. The value of d_thr=10mm is based on the fact that the effective radius of action of physiotherapy techniques is usually 5~15mm. This threshold is sufficient to accommodate the small spatial picking error introduced by mouse clicks (usually about 3~8mm), while ensuring that the operation point falls within the effective area. Nearest neighbor search is implemented using KD-Tree, with an average time complexity of O(log N) for a single query, ensuring a smooth interactive interface (actual single query <1ms). When the condition is met, p_op is added to the technique point list and displayed in blue; when the condition is not met, a prompt dialog box pops up to reject the operation. S=1 corresponds to a strict dual constraint (formal clinical physiotherapy), which forcibly restricts the operation point to the intersection of both human body region affiliation and robot accessibility, eliminating the risk of subsequent execution failure from the planning level. S=0 corresponds to a single accessibility constraint (offline debugging), allowing the operator to test the robot arm's limit workspace, facilitating system calibration and teaching programming. This dual-layer safety state design draws on the "safety level / programming level" permission separation concept in industrial collaborative robots and applies it to the field of physiotherapy planning for the first time, achieving a balance between clinical safety and debugging flexibility.

[0041] In this embodiment, the system performance was evaluated in a laboratory setting on 10 healthy subjects (5 males and 5 females, with a BMI range of 18.5 to 28.0). The subjects were in a prone position, and the camera's field of view covered the back to the lumbosacral region. The main test results are as follows: 3D positioning tests were conducted on 50 known marker points, with an average positioning error of 1.8mm (σ=0.4mm), meeting the design target of <2mm; the average intersection-union ratio (mIoU) of YOLOv9 instance segmentation reached 94.3%, and the single-frame inference time was 42ms; the batch LM inverse solution determination of 1500 candidate points took 138ms, with an average of 23 iterations per point to converge, and the average Jacobian condition number κ(J) was 18.4, far below the threshold upper limit of 100, indicating that the solution process is stable and far from singular configurations; the total end-to-end single-frame time was 187ms (42ms+138ms+7ms intersection), corresponding to a refresh rate of about 5.3Hz, which fully meets the real-time interaction requirements of the operation planning stage (the human eye's perception threshold for dynamic changes is about 10Hz, and 5Hz is smooth enough in static planning scenarios).

[0042] Compared to the control scheme without conditional number constraints, this embodiment, by introducing Jacobi conditional number constraints, effectively suppresses abrupt changes in joint velocity when the distal end approaches a singular configuration, reducing joint acceleration RMS by approximately 30%. In 200 simulated physiotherapy trajectory executions, the singular configuration protection shutdown was triggered zero times. In the safety mode effectiveness verification, 20 operators conducted a blind test: with S=0 (accessibility constraints only), the probability of mistakenly setting points in infeasible areas outside the body was 12.3%; after enabling strict double constraints with S=1, this probability dropped to 0%, and the average planning time for completing the 5-point scheme decreased by 42%. The operator's subjective score (out of 10) increased from 6.2 to 9.1, indicating that the visual feedback and safety constraint mechanism of this invention significantly improves planning efficiency and user confidence.

[0043] Example 2 This embodiment provides a physiotherapy area executable computing display system for implementing the method described in Embodiment 1, characterized in that it includes: The depth camera module is used to acquire RGB images and depth information of the human body; The visual processing module loads the YOLO v9 instance segmentation model and performs point cloud filtering and coordinate back projection. The robot control module stores DH parameters and joint limits, and performs inverse LM calculations and reachability determination. The intersection calculation module calculates H_base∩R_base using voxel hashing; the human-computer interaction module performs perspective projection rendering and KD-Tree nearest neighbor constraint determination. The safety mode switch is bound to a Boolean variable S∈{0,1} and controls the range of point constraints.

[0044] Example 3 This embodiment of a physiotherapy robot includes a robotic arm, an end-effector, an RGB-D camera, a controller, and a memory. The controller is used to execute the method described in Embodiment 1, calculate and display the intersection of the robot's working area and the human body area in real time, and dynamically constrain the operator's position settings according to the safety mode switch status.

[0045] The above-described embodiments are merely illustrative of certain implementations of the present invention, and are described in a relatively specific and detailed manner. However, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements are all within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for performing calculations and displaying data in a physiotherapy area, characterized in that, Includes the following steps: S1. Human contour recognition stage: The human body RGB image and depth map are acquired by the depth camera. The human body mask is output by the YOLOv9 instance segmentation network. The coordinate back projection of the pixels in the mask is performed according to the depth camera intrinsic parameter matrix K and the depth value D(u,v). After statistical outlier removal filtering, the human contour point set H_base is obtained. S2, Robot workable area calculation stage: The transformation matrix T_cam→base∈SE(3) obtained by hand-eye calibration is used to transform the camera coordinate system point set to the robot base coordinate system. For each point in the candidate point set after voxel sampling, the target pose with surface normal vector alignment constraint is constructed. The inverse kinematics is solved by Levenberg-Marquardt numerical iteration, and the joint limit constraint and Jacobian matrix condition number constraint κ(J(q))<κ_max are verified at the same time. The points that pass all constraints are marked as reachable points, forming the robot workable area R_base; S3, Intersection Region Calculation Stage: Based on the voxel hash table, perform set intersection calculation on H_base and R_base in O(N) time complexity to obtain the physiotherapy executable intersection region I=H_base∩R_base; S4, Human-computer interaction interface feedback stage: By observing the matrix V_view and the perspective projection matrix P_proj, each point in I is mapped to screen coordinates and rendered in the form of a high-brightness cloud on the 3D human-computer interaction interface. S5, Safety Mode Switching Constraint Phase: Set the safety mode state variable S∈{0,1}. When S=1, the operator's selected method candidate point p_op must satisfy min{‖p−p_op‖:p∈I}≤d_thr; when S=0, it must satisfy min{‖p−p_op‖:p∈R_base}≤d_thr. Those who do not meet the conditions will be refused operation and a prompt will be given.

2. The method for performing calculations and displaying information in a physiotherapy area according to claim 1, characterized in that: Step S1 includes S11. Image Acquisition and Preprocessing: The depth camera acquires color images I_rgb∈R^(H×W×3) and depth maps D∈R^(H×W) at 30fps. CLAHE contrast equalization is performed on I_rgb; 3×3 median filtering is performed on D to eliminate depth noise. S12, Instance Segmentation: Input the preprocessed I_rgb into the YOLOv9 instance segmentation network (pre-trained with COCO, fine-tuned with a special dataset), outputting a human instance mask M∈{0,1}^(H×W), with a confidence threshold τ_conf=0.

75. After performing morphological closing operations on the mask, downsampling is performed to obtain the set of boundary pixels: U={(u,v)|M(u,v)=1}; S13, Coordinate Backprojection: Using the depth camera intrinsic parameter matrix K and the depth map, each pixel (u,v) in U is backprojected to the camera coordinate system according to the pinhole camera model: [xc,yc,zc]ᵀ=D(u,v)·K⁻¹·[u,v,1]ᵀ Where K = [[fx,0,cx],[0,fy,cy],[0,0,1]]; S14. Validity Filtering and Outlier Removal: Invalid points with D(u,v)=0 or D(u,v)>1.5m are removed. Statistical Outlier Removal (SOR) is applied to the remaining point set with the number of neighborhood points k_sor=30 and the standard deviation factor λ_sor=2.

0. Finally, the human point set H_cam in the camera coordinate system is obtained.

3. The method for performing calculations and displaying data in a physiotherapy area according to claim 1, characterized in that: Step S2 includes S21, Coordinate System Transformation: Obtain the homogeneous transformation matrix T_cam→base∈SE(3) from the camera coordinate system to the robot base coordinate system through hand-eye calibration, and transform H_cam to the robot base coordinate system: [xb,yb,zb,1]ᵀ=T_cam→base·[xc,yc,zc,1]ᵀ The resulting point set is denoted as H_base. Using voxel side length δ=5mm for uniform sampling, and taking the centroid of each voxel, a candidate point set P={p1,p2,…,pN} is obtained. S22, Target Pose Construction: For each point p=(xb,yb,zb)ᵀ in P, estimate the human body surface normal vector n∈R³ through local point cloud PCA, and determine the tool head target pose R_tool∈SO(3), requiring the end z-axis to be aligned with the normal vector: arccos(nᵀ·R_tool·ez)<θ_max,θ_max=5°; S23, LM Inverse Kinematics Solution: The objective is to find a joint angle q such that FK(q) = T_target, while simultaneously satisfying the joint constraint qi_min ≤ qi ≤ qi_max, the singularity avoidance constraint κ(J(q)) < 100, and the convergence accuracy constraints ‖Δp‖ < 1 mm, ‖Δφ‖ < 0.5°. The joint update amount in the k-th iteration is: Δq⁽ᵏ⁾=−(JᵀJ+λ⁽ᵏ⁾I)⁻¹·Jᵀ·e⁽ᵏ⁾ Where e⁽ᵏ⁾∈R 6 Let λ be the end pose error vector, λ⁽ᵏ⁾ be the adaptive damping coefficient (initially 0.01, adjustment factor β=10), k_max=100, and add it to R_base if all constraints are satisfied, otherwise discard it.

4. The method for performing calculations and displaying information in a physiotherapy area according to claim 1, characterized in that: Step S3 specifically involves: In discrete point set representation, the intersection region for physical therapy is defined as: I=H_base∩R_base={p∈P|p∈H_base∧p∈R_base} Spatial hashing is used to accelerate intersection calculation. The time complexity of set member testing is O(1), and the total time complexity of intersection of the entire set is O(N). The voxel hash function is defined as: key(p,δ)=⌊px / δ⌋·P1⊕⌊py / δ⌋·P2⊕⌊pz / δ⌋·P3 Where P1=73856093, P2=19349663, P3=83492791 are preset large prime numbers, ⊕ represents bitwise XOR, and ⌊·⌋ represents rounding down.

5. The method for performing calculations and displaying information in a physiotherapy area according to claim 1, characterized in that: Step S4 includes S41, Perspective Projection: Each 3D point in the intersection point set I is mapped to the homogeneous clipping space after transformation by the view matrix V_view and the perspective projection matrix P_proj. [uc,vc,w]ᵀ=P_proj·V_view·[xb,yb,zb,1]ᵀ The screen pixel coordinates obtained by perspective division and viewport transformation are: xs = (uc / w + 1)·W / 2, ys = (vc / w + 1)·H / 2; S42. Mask Rendering and Host Computer Display: The system overlays the calculated intersection region I and the non-workable region onto the color image of the depth camera in the form of a semi-transparent mask, and displays it in real time through the host computer APP interface. The workable region (I) is highlighted with a green mask (RGB(0,255,0), α=0.75), and the non-workable region is covered with a red semi-transparent mask, so that the operator can intuitively identify the workable and non-workable areas on the patient's body surface. The screen radius of each point in the mask is adaptive according to the observation depth: r_screen = r_world·fy_proj / z_view, where r_world=δ / 2.

6. The method for performing calculations and displaying information in a physiotherapy area according to claim 1, characterized in that: Step S5 includes S51. Operation point acquisition: The operator sets candidate points on the 3D interface using the mouse, and the system obtains the candidate points p_op∈R³ through ray casting. S52. Safety Mode Criterion: The Boolean variable S∈{0,1} is bound to the interface checkbox, and the system applies real-time constraints based on the following criteria: When S=1: min{‖p − p_op‖ : p ∈ I} ≤ d_thr = 10mm; When S=0: min{‖p − p_op‖ : p ∈ R_base} ≤ d_thr = 10mm; Nearest neighbor search is implemented using a KD-Tree, with a single query time complexity of O(log N). When the conditions are met, p_op is added to the manipulation point list and displayed in blue; when the conditions are not met, a prompt dialog box pops up to reject the operation. S=1 corresponds to strict double constraints (formal clinical physiotherapy), and S=0 corresponds to a single reachability constraint (offline debugging).

7. A physiotherapy area computational display system for implementing the method of any one of claims 1 to 6, characterized in that, include: The depth camera module is used to acquire RGB images and depth information of the human body; The visual processing module loads the YOLO v9 instance segmentation model and performs point cloud filtering and coordinate back projection. The robot control module stores DH parameters and joint limits, and performs inverse LM calculations and reachability determination. The intersection calculation module calculates H_base∩R_base using voxel hashing; the human-computer interaction module performs perspective projection rendering and KD-Tree nearest neighbor constraint determination. The safety mode switch is bound to a Boolean variable S∈{0,1} and controls the range of point constraints.

8. A physiotherapy robot, characterized in that: It includes a robotic arm, an end-effector, an RGB-D camera, a controller, and a memory. The controller is used to execute the method of any one of claims 1 to 6, calculate and display the intersection of the robot's work area and the human body area in real time, and dynamically constrain the operator's position settings according to the safety mode switch status.

Images