A medical human-computer cooperation-oriented human body motion prediction and surgical area invasion early warning method and device, a storage medium and an electronic device

Abstract

The application discloses a medical human-computer cooperation-oriented human body motion prediction and operation area invasion early warning method and device, a storage medium and electronic equipment, and belongs to the technical field of medical robot safety. The method aims at the problems that the sudden intrusion of limbs into the operation area in the cooperation of the surgical robot is easy to cause collision, the existing prediction model has long-time drift, physical distortion and monocular deployment difficulty, input features are constructed through S-G filtering and kinematic feature injection, the Informer network is improved by using end frame filling and damping residual structure, the biomechanical rationality of the prediction result is constrained in combination with the physical inverse normalization joint loss, real-time operation area invasion early warning under the monocular camera is realized through EMA filtering and uncalibrated pseudo-projection. The long-time prediction accuracy of the application is better than that of mainstream methods, the operation area invasion can be early warned by 1000ms, the early warning accuracy rate reaches 95.23%, the camera does not need to be calibrated and can be deployed, and is suitable for active safety protection of medical scenes such as surgical robots and rehabilitation robots.

Description

Technical Field

[0001] This invention relates to the field of medical robot safety technology, and in particular to a method, device, storage medium and electronic device for predicting human motion and warning of surgical area intrusion in medical human-machine collaboration, which is applicable to active safety protection of human-machine collaboration in medical scenarios such as surgical robots and rehabilitation robots. Background Technology

[0002] With the rapid development of robotics in the medical field, human-machine collaboration is becoming a core supporting technology for modern surgery, rehabilitation therapy, and clinical nursing. In robot-assisted surgery, surgeons and surgical robots share the same workspace, and real-time, accurate human motion prediction capabilities are crucial for ensuring surgical safety, achieving natural interaction, and avoiding collisions. Unlike the structured environments of industrial settings, medical operating rooms are highly dynamic and unstructured: the movements of medical personnel are highly random and vary from person to person, and the frequent changes in surgical instruments and patient positions place extremely high demands on the perception system. Furthermore, in rehabilitation training and elderly care scenarios, human motion prediction technology can also help nursing robots intervene in a timely manner by identifying abnormal movement patterns (such as signs of impending falls), thus reducing the risk of medical accidents.

[0003] Existing methods for predicting human motion can be mainly divided into three categories: methods based on recurrent neural networks (RNN / LSTM / GRU), methods based on graph convolutional networks (GCN), and methods based on Transformers. RNN-based methods capture temporal dependencies through gating mechanisms, but suffer from error accumulation and gradient vanishing problems in long-sequence predictions, and are difficult to parallelize, resulting in insufficient real-time performance. GCN-based methods model the skeletal spatial topology through graph structures, but most use fixed graph structures, making it difficult to adapt to dynamic joint relationships, and their long-term modeling capabilities are insufficient. Transformer-based methods capture long-term dependencies through self-attention mechanisms, but standard Transformers suffer from quadratic complexity, resulting in high computational overhead, and the purely data-driven paradigm lacks prior physical constraints on human kinematics, easily leading to physically unreasonable results such as changes in bone length and joint angles exceeding physiological ranges.

[0004] More critically, existing methods suffer from the following core flaws when applied in medical scenarios: First, long-term prediction suffers from the "mean regression" problem. Using zero-filling to initialize the decoder leads to rapid deceleration and stiffening at the average posture in the later stages of prediction, making it impossible to accurately predict sudden surgical intrusions. Second, the prediction results lack physical plausibility. The loss function of the normalized space can easily lead to skeletal deformities and motion jitter, which does not conform to the laws of human biomechanics. Third, monocular vision is sensitive to noise. Depth noise in monocular pose estimation is amplified by differential operations, resulting in jitter in the prediction results and a high false alarm rate. Fourth, deployment costs are high. Most existing methods require multi-camera motion capture systems or strict camera intrinsic parameter calibration, making them difficult to directly adapt to existing clinical monocular cameras and thus difficult to implement. In addition, existing safety systems in medical scenarios are mostly post-event collision detection, which cannot predict intrusion risks in advance, easily leading to sudden stops of the robotic arm that interrupt the surgical procedure or even cause medical accidents.

[0005] Therefore, there is an urgent need for a human motion prediction and surgical area early warning method that can be adapted to monocular uncalibrated cameras, has high prediction accuracy, strong physical rationality, and good real-time performance, so as to achieve proactive safety protection in medical human-machine collaboration. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, device, storage medium and electronic device for predicting human motion and early warning of surgical area invasion in medical human-machine collaboration. It solves the problems of long-term prediction drift, physical distortion, monocular noise sensitivity, high deployment cost and inability to provide early warning in the prior art. It achieves real-time, high-precision and physically reasonable human motion prediction under uncalibrated monocular camera and provides early warning of surgical area invasion risk 1000ms in advance.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, this invention provides a method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration, comprising four core steps: First, data preprocessing is performed to acquire the surgical scene video stream captured by a monocular RGB camera and extract the human body 3D joint coordinate sequence; coordinate alignment is performed with the pelvic joint as the root node to eliminate global translation components, and the joint coordinate sequence is denoised using a Savitzky-Golay filter to suppress inter-frame noise in monocular measurements; based on the denoised position sequence, the instantaneous velocity and instantaneous acceleration of the joint are calculated, and the higher-order kinematic features of position, velocity, and acceleration are cascaded in the channel dimension and normalized by Z-Score to obtain the model input feature sequence, thereby enhancing the model's ability to perceive motion inertia.

[0009] Then, physical guided residual prediction is performed. The input feature sequence is fed into a pre-trained physical guided residual Informer network (PRISM). This network uses the fully self-attention Informer as its backbone, retaining the linear complexity advantage of ProbSparse attention while restoring fully self-attention to avoid loss of kinematic information. During decoder initialization, the traditional zero-padding method is abandoned, and the joint coordinates of the last historical frame are used to fill the future time slots to avoid regression of the prediction results to the mean. The model prediction target is set as the residual displacement of each future frame relative to the last historical frame, rather than the absolute coordinates, to reduce long-term cumulative drift. A time decay damping coefficient is introduced into the residual connection to simulate the natural kinetic energy consumption of human skeletal muscle movement and suppress spatial drift under high dynamic motion. Finally, the normalized 3D joint coordinate prediction results within the preset future time are output.

[0010] During the model training phase, physical constraint optimization is performed: the normalized coordinates output by the network are denormalized to the real millimeter-level physical space in real time to avoid geometric distortion caused by normalization compression; a joint loss function is designed, which includes time-weighted position loss, skeletal rigidity loss, and velocity smoothing loss: the time-weighted position loss gives higher weight to the last frame in long time to alleviate long-term prediction drift; the skeletal rigidity loss is based on the topological constraint of human skeleton to keep the distance between adjacent joints constant to avoid skeletal distortion; the velocity smoothing loss constrains the temporal continuity of the motion trajectory to eliminate prediction jitter. The network is guided to learn motion characteristics that conform to biomechanical laws through the joint use of multiple physical losses.

[0011] Finally, an intrusion warning for the surgical area is implemented: the predicted 3D joint coordinates are filtered and denoised using an exponential moving average (EMA) filter to smooth the high-frequency noise of depth in monocular pose estimation; based on the dynamic ratio of the average shoulder width in human anatomy to the pixel shoulder width in the current image, a projection scaling factor is calculated, which can pseudo-project the 3D joint coordinates into 2D pixel coordinates without camera intrinsic parameter calibration; the rectangular pixel range of the surgical area is pre-calibrated, and it is determined in real time whether the 2D pixel coordinates of high-risk joints such as bilateral wrists fall into the surgical area within the next 1000ms. If there is an intrusion risk, an early warning is triggered, and the surgical robot is controlled to decelerate or stop suddenly to achieve active safety protection.

[0012] Compared with existing technologies, this invention has the following advantages: 1. High long-term prediction accuracy: By solving the mean regression problem through last-frame padding and suppressing long-term drift through damped residual structures, the prediction error at 400ms on the Human3.6M dataset is reduced by 14.5% compared to the state-of-the-art method, and the prediction error at 1000ms reaches 108.3mm, which is superior to mainstream comparison methods; 2. Strong physical rationality: By using physical inverse normalization joint loss to constrain constant bone length and smooth motion, the bone length error is reduced from 17.7mm to 9.3mm, eliminating bone distortion and jitter problems in the prediction results, which conforms to the laws of human biomechanics; 3. Low monocular deployment cost: No camera intrinsic parameter calibration is required. The pseudo-projection method based on anatomical shoulder width can be directly adapted to existing clinical pendant monocular cameras. Monocular noise is suppressed through SG filtering and EMA filtering, resulting in strong robustness; 4. Good real-time performance: Based on the improved Informer backbone network, single-batch inference only requires 14ms, meeting the real-time processing requirements of 25FPS video streams; 5. Excellent early warning performance: In simulated surgical scenarios, the accuracy rate of early warning 1000ms in advance reaches 95.23%, and the accuracy rate of short-term early warning 400ms reaches 92.61%. It can predict 73% of the intrusion risk 1 second in advance, with a low false alarm rate, effectively avoiding the surgical robot from stopping unexpectedly, and realizing a safety paradigm upgrade from post-collision response to pre-event proactive early warning. Attached Figure Description

[0013] Figure 1 A flowchart of the surgical area intrusion early warning system provided in an embodiment of the present invention.

[0014] Figure 2 This is a data processing flowchart provided for an embodiment of the present invention.

[0015] Figure 3 The diagram shows the overall architecture of the PRISM network provided in this embodiment of the invention.

[0016] Figure 4 This is a schematic diagram of the experimental environment for a simulated surgical scenario provided in an embodiment of the present invention.

[0017] Figure 5 This is a schematic diagram illustrating the prediction results on the Human3.6M dataset provided in this embodiment of the invention.

[0018] Figure 6 This is a comparison chart of prediction errors between the present invention and mainstream methods, provided in an embodiment of the invention.

[0019] Figure 7 This is a schematic diagram illustrating the surgical area intrusion early warning effect provided in an embodiment of the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited to the following embodiments.

[0021] Example 1: This example provides a method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration, such as... Figure 1-3 As shown, the specific steps include:

[0022] S1: Data Preprocessing. The process for this step is as follows: Figure 2 As shown, specifically: S1.1: Video Acquisition and Pose Extraction: Surgical scene videos were acquired using a clinically common pendant monocular RGB camera with a resolution of 1920×1080 and a frame rate of 29.9 FPS. The videos were resampled to 25 FPS (frame interval 40ms). The MediaPipe Pose pose estimation model was used to extract the 3D coordinates of 17 core human joints from each frame of video. The joints included the pelvis, left hip, right hip, left knee, right knee, left ankle, right ankle, spine, thoracic cavity, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, neck, and head. S1.2: Root Node Alignment: Using the pelvic joint as the root node, the 3D coordinates of all joints were subtracted from the 3D coordinates of the pelvic joint to eliminate the global translation component of the human body in the world coordinate system. This ensured that the input features were translation-invariant and avoided the influence of camera position changes on the prediction. S1.3: SG Filtering Denoising: A Savitzky-Golay (SG) low-pass filter is used to perform time-domain filtering on the aligned joint coordinate sequence. The filter window size is set to 5 frames. A third-order polynomial is used for local least-squares fitting to retain the peak values ​​of high-dynamic joint motion while suppressing inter-frame measurement noise from monocular camera pose estimation, thus avoiding noise amplification in subsequent difference operations. S1.4: Kinematic Feature Injection: For the denoised position sequence, the instantaneous velocity vector of the joint is calculated using first-order finite difference on the time axis, and the instantaneous acceleration vector is calculated using second-order finite difference. For the boundary cases of the first frame of the sequence, zero forward difference is used for filling. The three types of features—position, velocity, and acceleration—are concatenated in the channel dimension, expanding the feature dimension of a single frame from the original 3×17=51 dimensions to 9×17=153 dimensions, enabling the model to perceive motion inertia and improving short-term prediction accuracy. S1.5: Normalization: Perform Z-Score normalization on the concatenated feature sequence, calculate the mean and standard deviation of the training set features, and standardize the features to a distribution with a mean of 0 and a variance of 1, thereby accelerating network training convergence.

[0023] S2: Physically Guided Residual Prediction (PRISM Network Inference). The PRISM network architecture used in this step is as follows: Figure 3 As shown, the network uses an improved Informer as its backbone. The specific inference process is as follows: S2.1: Network structure settings: The encoder and decoder of the PRISM network are both set to 5 layers, and the hidden layer dimension is... =256, the hidden layer dimension of the feedforward network =2048, Number of attention heads =8, dropout rate set to 0.1; replace the original Informer's ProbSparse self-attention with full self-attention to avoid the loss of local kinematic information caused by sparse attention, while retaining the non-autoregressive decoding characteristics of Informer to ensure inference speed. S2.2: Input and initialization: input the normalized feature sequence of 50 frames (corresponding to 2 seconds of observation time) obtained in S1 into the encoder; the decoder input is divided into two parts, the first part is the features of the last 10 frames of the observation sequence, and the second part is the time slots corresponding to the next 25 frames (1000ms); abandon the traditional zero-filling method, replace all the placeholders of the future time slots with the joint features of the last frame of the observation sequence (i.e., last frame filling), avoid the "mean regression" problem caused by the zero value corresponding to the mean of the data, so that the prediction result regresses to the average pose, and the model maintains inertial motion when there is no new motion intention. S2.3: Damped Residual Prediction: The model does not directly predict the absolute coordinates of future frames, but rather predicts the residual displacement at each future time step relative to the last historical frame (the last frame of the observation sequence); when the residuals are accumulated to obtain the final coordinates, a time-decayed damping coefficient is introduced. =0.995, which is the residual displacement multiplied by the value at the t-th prediction time step. It simulates the natural kinetic energy consumption during human skeletal muscle movement. As the prediction time increases, the weight of the residual displacement decreases exponentially, effectively suppressing abnormal spatial drift under high dynamic postures in long-term prediction and ensuring absolute continuity between the first frame prediction and the last historical frame. S2.4: Output: The network outputs the normalized 3D joint coordinate prediction results for the next 25 frames (1000ms).

[0024] S3: Model Training and Physical Constraint Optimization. The training process of the PRISM network is as follows: S3.1: Training Dataset: The Human3.6M public dataset and a self-built simulated surgery dataset were used for training. The Human3.6M dataset used all actions of subjects S1, S5, S6, S7, and S8 as the training set, and S9 and S11 as the test set, downsampled to 25 FPS, and extracted 17 core joints. The self-built simulated surgery dataset contained 11 simulated surgical operation videos, totaling approximately 10,000 frames. The first 9 videos were used as the training set, and the last 2 as the test set. S3.2: Training Parameter Settings: The optimizer used AdamW, with an initial learning rate of 1e-4, a weight decay coefficient of 1e-4, and a cosine annealing restart scheduler to adjust the learning rate. The restart period T0=10, the batch size was 64, and the training lasted for 50 epochs, completed on a single NVIDIA RTX 4070 SUPER graphics card. S3.3: Physical Inverse Normalization: Before calculating the loss, the normalized coordinates of the network output are inversely normalized to the real millimeter-level physical space in real time using the mean and standard deviation of the training set. This avoids geometric distortion caused by normalization compressing the physical variance of different joints. S3.4: Joint Loss Calculation: The joint loss function is calculated in the real physical space, and the total loss is calculated. ,in, =0.5, =0.1: Time-weighted location loss Assign weights to each prediction time step t The scale adjustment factor is set to =1.5, Representing the predicted future frame, the weight of the time step increases with distance from the observation time, reaching a maximum weight of more than four times that of the final frame. The weighted L2 distance between the predicted and true coordinates is calculated, focusing the optimization efforts on the positional accuracy of the final frame over long periods, thus mitigating long-term drift; skeletal rigidity. The loss is based on the topology of human anatomy and skeleton, defining 16 bone connection edges (such as pelvis-left hip, left hip-left knee, etc.), calculating the L2 distance between the predicted bone length and the actual bone length, penalizing bone length deviations, constraining bone length to be constant, and avoiding bone deformities caused by joint displacement; velocity smoothing loss. Calculate the L2 distance between the first-order velocity difference of the predicted sequence and the actual velocity difference to constrain the temporal continuity of the motion trajectory and eliminate single-frame prediction jitter. S3.5: Backpropagation: Calculate the gradient based on the joint loss and backpropagate to update the network parameters until the network converges.

[0025] S4: Surgical area intrusion warning. The warning process for this step is as follows: Figure 1As shown, specifically: S4.1: EMA filtering for noise reduction: The 2D joint coordinates extracted by MediaPipe and the 3D joint coordinates predicted by PRISM are smoothed using an exponential moving average (EMA) filter, with an inertia weight α=0.8, i.e., the current smoothed value = 0.8 × historical smoothed value + 0.2 × current measurement value. This effectively filters out high-frequency noise in the depth direction of monocular pose estimation and avoids false alarms caused by predicted skeleton jitter. S4.2: Dynamic pseudo-projection: Without camera intrinsic parameter calibration, based on human anatomy priors, the average shoulder width of an adult is 380mm; the pixel distance between the shoulders in the current frame is detected in real time, and the dynamic projection scaling factor s = 380mm / pixel distance between the shoulders is calculated; the 3D joint coordinates predicted by PRISM are aligned from the upward Z-axis in 3D space to the downward Y-axis in the image plane through coordinate transformation, and the 3D physical displacement is converted into 2D pixel displacement by the scaling factor s, which is superimposed on the 2D joint coordinates of the current frame to obtain the 2D pixel coordinates of the future frame. S4.3: Surgical Area Marking and Intrusion Judgment: The user manually marks the core surgical area (surgical area) as a rectangular range in the video screen and records the coordinates of the upper left corner (x1, y1) and the lower right corner (x2, y2); the user focuses on monitoring the bilateral wrist joints (high-risk end effectors of surgical operations). For the wrist coordinates (x, y) of each future time step, if x1≤x≤x2 and y1≤y≤y2, it is determined that there is an intrusion risk in that time step. S4.4: Warning Trigger: The surgical area intrusion warning is triggered when any of the following conditions are met: (1) The wrist coordinates of the current frame have fallen into the surgical area; (2) There is an intrusion risk in any predicted time step within the next 1000ms; After the warning is triggered, a deceleration or emergency stop signal is sent to the surgical robot control system, and the surgical area is marked in red in the video screen to alert medical staff.

[0026] Example 2: This example provides an apparatus corresponding to the method in Example 1, including: a data preprocessing module, used to execute step S1 in Example 1, completing video acquisition, pose extraction, root node alignment, SG filtering, kinematic feature injection, and normalization; a motion prediction module, used to execute step S2 in Example 1, completing forward inference of the PRISM network, and outputting the predicted results of future 3D joint coordinates through last frame padding and damped residual structure; a model training module, used to execute step S3 in Example 1, completing dataset construction, training parameter setting, physical inverse normalization, joint loss calculation, and network parameter update; and an intrusion warning module, used to execute step S4 in Example 1, completing EMA filtering, dynamic pseudo-projection, surgical area calibration, intrusion determination, and warning triggering.

[0027] Example 3: This example provides a computer-readable storage medium having a computer program stored thereon, which when executed by a processor, implements the steps of the human motion prediction and surgical site intrusion warning method for medical human-machine collaboration described in Example 1. The storage medium includes various media capable of storing program codes such as U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk.

[0028] Example 4: This example provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, it implements the steps of the human motion prediction and surgical site intrusion warning method for medical human-machine collaboration described in Example 1. The electronic device can be a main control computer of a surgical robot, an edge computing box matched with an operating room pendant, or an independent medical safety warning terminal.

[0029] To verify the effect of the present invention, comparative experiments and ablation experiments are carried out, and the experimental results are as follows: On the Human3.6M dataset, the 80ms short-term prediction error of the method of the present invention is 8.3mm, which is 13.5% lower than 9.6mm of AMHGCN; the 400ms prediction error is 49.2mm, which is 14.5% lower than 57.6mm of AMHGCN; the 1000ms long-term prediction error is 108.3mm, which is better than 109.4mm of siMLPe; on difficult actions such as SittingDown, the 1000ms prediction error is 19.5% lower than that of siMLPe. The single-batch inference time of the model is 14ms, which is 67.4% shorter than that of the PGBIG method, meeting the real-time requirements. In the simulated surgical scenario, the recall rate of the method of the present invention for early warning 80ms in advance is 84.37%, and the precision rate is 87.00%; the recall rate for early warning 1000ms in advance is 72.87%, the precision rate reaches 95.23%, the F1 value is stable above 80, the false alarm rate is low, and it will not cause frequent false emergency stops of the robot. The ablation experiment shows that: last-frame filling reduces the bone error from 17.7mm to 9.3mm, the damping coefficient reduces the peak error from 39.2mm to 34.8mm; kinematic feature injection reduces the 80ms starting error from 15.9mm to 8.3mm; EMA filtering increases the 400ms warning recall rate from 50.56% to 79.79%. Each module brings significant performance improvement.

[0030] The above description is only preferred embodiments of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration, characterized in that, include: S1. Data preprocessing steps: acquire the surgical scene video stream captured by a monocular RGB camera and extract the human body 3D joint coordinate sequence; Coordinate alignment is performed with the pelvic joint as the root node, and the joint coordinate sequence is denoised using a Savitzky-Golay filter. The instantaneous velocity and instantaneous acceleration of the joint are calculated based on the denoised position sequence. The position, velocity, and acceleration features are concatenated in the channel dimension and normalized to obtain the model input feature sequence. S2. Physically Guided Residual Prediction Step: The input feature sequence is input into a pre-trained Physically Guided Residual Informer network PRISM. The network uses an Informer with a fully self-attention mechanism as its backbone. During decoder initialization, the joint coordinates of the last historical frame are used to fill the future time slots. The residual displacement of each future frame relative to the last historical frame is used as the prediction target. A time decay damping coefficient is introduced into the residual connection to output the normalized 3D joint coordinate prediction results within the preset future time. S3. Physical Constraint Optimization Step: When training the PRISM network, the normalized coordinates output by the network are denormalized to the real physical space. The network training is constrained by a joint loss function, which includes time-weighted position loss, bone rigidity loss, and velocity smoothing loss. S4. Surgical Area Intrusion Warning Step: The predicted 3D joint coordinates are denoised by exponential moving average (EMA) filtering. A dynamic projection scaling factor is calculated based on the ratio of the average shoulder width of the human body to the pixel shoulder width in the image. The 3D joint coordinates are pseudo-projected into 2D pixel coordinates. It is determined whether the 2D pixel coordinates of high-risk joints within a preset time period fall within the pre-defined surgical area pixel range. If so, a surgical area intrusion warning is triggered.

2. The method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration according to claim 1, characterized in that, The step S1 of extracting the human body 3D joint coordinate sequence specifically involves: extracting the 3D coordinate sequence of 17 core human joints from monocular RGB video frames using the MediaPipe pose estimation model, and resampling the video to 25FPS.

3. The method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration according to claim 1, characterized in that, The root node alignment in step S1 specifically involves subtracting the three-dimensional coordinates of the pelvic joint from the three-dimensional coordinates of all joints to eliminate the global translation component of the human body and achieve coordinate translation invariance. The Savitzky-Golay filter uses a 5-frame window and a 3rd-order polynomial for local least-squares fitting to suppress inter-frame measurement noise while preserving motion peaks.

4. The method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration according to claim 1, characterized in that, The calculation of instantaneous velocity and instantaneous acceleration in step S1 is as follows: the instantaneous velocity is obtained by performing a first-order finite difference on the time axis of the denoised position sequence, and the instantaneous acceleration is obtained by performing a second-order finite difference. The first frame is filled with zero forward difference. After concatenating the position, velocity and acceleration features, the feature dimension of a single frame is expanded from 3N to 9N, where N is the number of joints.

5. The method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration according to claim 1, characterized in that, In step S2, the final frame padding specifically involves replacing all placeholders in the decoder input corresponding to future time steps with joint features from the last frame of the historical observation sequence to avoid the problem of prediction results regressing to the data mean due to zero padding. The time decay damping coefficient is set to 0.995, which exponentially decays the residual displacement as the prediction time step increases, simulating the natural kinetic energy consumption of human movement and suppressing spatial drift in long-term predictions.

6. The method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration according to claim 1, characterized in that, The time-weighted position loss in step S3 specifically involves assigning higher loss weights to future time steps that are further away from the observation time, with the weights increasing exponentially with the time step and a scale adjustment factor of 0.1, focusing the optimization efforts on the positional accuracy of the long-term prediction endpoint. The skeletal rigidity loss specifically involves penalizing the deviation between the distance between adjacent joints in the prediction result and the actual bone length based on the topological connectivity of the human skeleton, thus constraining the bone length to be constant. The velocity smoothing loss specifically involves calculating the mean square error of the prediction sequence in the first derivative domain, constraining the temporal continuity of the motion trajectory, and eliminating prediction jitter.

7. The method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration according to claim 1, characterized in that, In step S3, the weights of the joint loss function are set as follows: time-weighted position loss weight is 1.0, skeletal rigidity loss weight is 0.1, and velocity smoothing loss weight is 0.

1.

8. The method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration according to claim 1, characterized in that, In step S4, the inertial weight of the EMA filter is 0.8, which is used to smooth the high-frequency noise in the depth direction of monocular pose estimation; the dynamic projection scaling factor is the ratio of the actual average shoulder width of the human body (380mm) to the pixel distance between the shoulders detected in the current frame; the pseudo-projection specifically involves converting the coordinates on the Z-axis in 3D space into pixel coordinates on the Y-axis in the image plane, and combining the projection scaling factor to convert the 3D physical displacement into 2D pixel displacement.

9. The method for predicting human motion and providing early warning of surgical area intrusion in medical human-machine collaboration according to claim 1, characterized in that, In step S4, the high-risk joints are the bilateral wrist joints; the pixel range of the surgical area is manually marked by the user as a rectangular area, with the upper left corner coordinates as (x1, y1) and the lower right corner coordinates as (x2, y2); when the 2D pixel coordinates of any wrist joint in any predicted future time step satisfy x1≤x≤x2 and y1≤y≤y2, it is determined that there is an intrusion risk and an early warning is triggered; The warning is triggered when: an intrusion has occurred in the current frame, or an intrusion is predicted to occur within the next 1000ms.

10. A human motion prediction and surgical area invasion early warning device for medical human-machine collaboration, characterized in that, include: The data preprocessing module is used to acquire the surgical scene video stream captured by a monocular RGB camera and extract the human body 3D joint coordinate sequence; Coordinate alignment is performed with the pelvic joint as the root node, and the joint coordinate sequence is denoised using a Savitzky-Golay filter. The instantaneous velocity and instantaneous acceleration of the joint are calculated based on the denoised position sequence. The position, velocity, and acceleration features are concatenated in the channel dimension and normalized to obtain the model input feature sequence. The motion prediction module is used to input the input feature sequence into a pre-trained Physically Guided Residual Informer Network (PRISM). The network uses an Informer with a fully self-attention mechanism as its backbone. During decoder initialization, the joint coordinates of the last historical frame are used to fill the future time slots. The residual displacement of each future frame relative to the last historical frame is used as the prediction target. A time decay damping coefficient is introduced into the residual connection to output the normalized 3D joint coordinate prediction results within a preset future time. The model training module is used to train the PRISM network. During training, the normalized coordinates output by the network are denormalized to the real physical space. The network training is constrained by a joint loss function, which includes time-weighted position loss, skeletal rigidity loss, and velocity smoothing loss. The intrusion warning module is used to denoise the predicted 3D joint coordinates using exponential moving average (EMA) filtering. Based on the ratio of the average shoulder width of the human body to the pixel shoulder width in the image, a dynamic projection scaling factor is calculated to pseudo-project the 3D joint coordinates into 2D pixel coordinates. The module determines whether the 2D pixel coordinates of high-risk joints within a preset time period fall within the pre-defined surgical area pixel range. If so, a surgical area intrusion warning is triggered.

11. The human motion prediction and surgical area invasion early warning device for medical human-machine collaboration according to claim 10, characterized in that, The PRISM network has 5 layers for both the encoder and decoder, 256 hidden layer dimensions, 2048 feedforward network dimensions, 8 attention heads, and a dropout rate of 0.

1. It uses the AdamW optimizer with an initial learning rate of 1e-4 and a weight decay of 1e-4. The learning rate is adjusted by restarting the scheduler through cosine annealing.

12. The human motion prediction and surgical area intrusion early warning device for medical human-machine collaboration according to claim 10, characterized in that, The device achieves a single-batch inference time of 14ms on a single NVIDIA RTX 4070 SUPER graphics card, meeting the real-time operation requirements at 25FPS.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for predicting human motion and early warning of surgical area intrusion as described in any one of claims 1-9.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the human motion prediction and surgical area intrusion early warning method for medical human-machine collaboration as described in any one of claims 1-9.

Images