Spinal surgery robot arm precision control system fusing deep learning and dynamic calibration

Abstract

The application relates to the field of artificial intelligence, and discloses a spine surgery mechanical arm precision regulation system combining deep learning and dynamic calibration, which comprises a digital twin construction module, a motion path analysis module, a coarse regulation module, a fine regulation module and a micrometer regulation module.The digital twin construction module is used for constructing a dynamic spine digital twin of a target object.The motion path analysis module is used for generating a planned motion path of the mechanical arm.The coarse regulation module is used for calculating a translation vector and a rotation vector of the mechanical arm when a path error is not less than a preset first threshold value, and performing coarse regulation of the mechanical arm.The fine regulation module is used for calculating a stiffness coefficient of the mechanical arm when the path error is less than the first threshold value and not less than a preset second threshold value, and performing fine regulation of the mechanical arm.The micrometer regulation module is used for calculating a pulse voltage required by a micro-motion platform when the path error is less than the second threshold value and not less than a preset safety threshold value, and performing micrometer regulation of the mechanical arm.The application can improve the control precision and safety of the spine surgery mechanical arm.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a precision control system for a spinal surgery robotic arm that integrates deep learning and dynamic calibration. Background Technology

[0002] A spinal surgery robotic arm is an intelligent robotic system used in spinal surgery. Through high-precision motion control, real-time navigation, and force feedback technologies, it assists surgeons in performing complex spinal surgical procedures such as pedicle screw implantation, vertebroplasty, and decompression fixation. The precise control of the spinal surgery robotic arm allows it to replicate the pre-planned ideal path onto the patient with sub-millimeter precision, ensuring that surgical tools strictly follow the preset path, thereby minimizing the risk of neurovascular damage and reducing surgical risks.

[0003] Traditional surgical robotic arm control methods use pre-programmed paths. By registering images before surgery, the pre-programmed path of the surgical robotic arm is generated based on the registered structure. However, this method cannot adapt to anatomical deformations caused by respiratory movements and instrument interaction forces in real time. As a result, it cannot effectively adjust when tissues are displaced during surgery, leading to large positioning errors during the operation. Summary of the Invention

[0004] This invention provides a precision control system for a spinal surgery robotic arm that integrates deep learning and dynamic calibration, with the main purpose of improving the control accuracy and safety of the spinal surgery robotic arm.

[0005] To achieve the above objectives, the present invention provides a precision control system for a spinal surgery robotic arm that integrates deep learning and dynamic calibration, comprising:

[0006] A digital twin construction module is used to acquire multimodal data of a target object, wherein the multimodal data includes CT image data, three-dimensional point cloud data, and instrument contact force data, in order to construct a dynamic spinal digital twin of the target object;

[0007] The motion path analysis module is used to calculate the dynamic deformation field of the spine corresponding to the target object based on the dynamic digital twin of the spine, so as to generate the planned motion path of the robotic arm corresponding to the spine.

[0008] The coarse-level control module is used to acquire the real-time motion path of the robotic arm and calculate the path error between the real-time motion path and the planned motion path. When the path error is not less than a preset first threshold, the module calculates the translation vector and rotation vector of the robotic arm to generate the pose calibration command of the robotic arm. Based on the pose calibration command, the module executes the coarse-level control of the robotic arm.

[0009] The precision control module is used to calculate the stiffness coefficient of the robotic arm when the path error is less than the first threshold and not less than the preset second threshold, determine the displacement compensation amount of the robotic arm based on the stiffness coefficient, generate the impedance calibration command of the robotic arm, and execute the precision control of the robotic arm based on the impedance calibration command.

[0010] The micrometer-level control module is used to calculate the pulse voltage required by the micro-motion platform corresponding to the robotic arm when the path error is less than the second threshold and not less than the preset safety threshold, so as to generate the micrometer-level calibration command of the robotic arm and execute the micrometer-level control of the robotic arm according to the micrometer-level calibration command.

[0011] Optionally, when the path error is not less than a preset first threshold, calculating the translation vector and rotation vector of the robotic arm includes:

[0012] Based on the path error, the actual pose and planned pose of the robotic arm are determined;

[0013] Calculate the translation vector of the robotic arm based on the actual pose and the planned pose;

[0014] Determine the actual pose four elements of the actual pose and the planned pose four elements of the planned pose, respectively;

[0015] The relative rotation quaternion of the robotic arm is calculated based on the actual pose quaternion and the planned pose quaternion.

[0016] Based on the relative rotation quaternion, calculate the rotation angle and rotation unit vector of the robotic arm;

[0017] The rotation vector of the robotic arm is determined based on the rotation angle and the rotation unit vector.

[0018] Optionally, calculating the stiffness coefficient of the robotic arm when the path error is less than the first threshold and not less than a preset second threshold includes:

[0019] Extract the CT grayscale values ​​and bone-device contact force from the multimodal data corresponding to the robotic arm;

[0020] Based on the CT grayscale values, analyze the current bone density of the spinal region corresponding to the robotic arm;

[0021] The stiffness coefficient of the robotic arm is calculated based on the current bone density in the region, the bone-device contact force, and the path error.

[0022] Optionally, determining the displacement compensation amount of the robotic arm based on the stiffness coefficient includes:

[0023] Based on the stiffness coefficient, analyze the damping coefficient of the robotic arm;

[0024] By combining the damping coefficient and the stiffness coefficient, an impedance model of the robotic arm is fitted.

[0025] Based on the path error corresponding to the robotic arm, the static compensation amount of the robotic arm is calculated using the impedance model.

[0026] Determine the respiratory motion pattern of the target object corresponding to the robotic arm;

[0027] Based on the breathing motion pattern and the static compensation amount, the displacement compensation amount of the robotic arm is analyzed.

[0028] Optionally, when the path error is less than the second threshold and not less than a preset safety threshold, calculating the pulse voltage required by the micro-motion platform corresponding to the robotic arm includes:

[0029] Determine the mapping coefficients between the robotic arm's coordinate system and the micro-motion platform's coordinate system;

[0030] Based on the path error and the mapping coefficient, the amount of displacement that the robotic arm needs to compensate is determined;

[0031] The displacement-voltage linear relationship of the micro-motion platform is analyzed to determine the displacement-voltage conversion coefficient of the micro-motion platform;

[0032] The pulse voltage of the micro-motion platform is determined based on the displacement-voltage conversion coefficient and the displacement amount.

[0033] Optionally, constructing a dynamic digital twin of the spine of the target object includes:

[0034] Construct an image coordinate system for the CT image data corresponding to the target object and a surgical three-dimensional coordinate system for the three-dimensional point cloud data of the CT image data corresponding to the target object, respectively;

[0035] Determine the transformation matrix between the image coordinate system and the surgical three-dimensional coordinate system;

[0036] Based on the transformation matrix, spatiotemporal registration is performed on the multimodal data corresponding to the target object to obtain multimodal registration data;

[0037] Multimodal features are extracted from the multimodal registration data to perform data fusion on the multimodal registration data, resulting in fused data.

[0038] Based on the fused data, a three-dimensional model of the spine of the target object is fitted;

[0039] The dynamic behavior of the three-dimensional spinal model is simulated to construct a dynamic digital twin of the spine.

[0040] Optionally, calculating the dynamic deformation field of the spine corresponding to the target object based on the dynamic digital twin of the spine includes:

[0041] The initial static equilibrium state of the spine is simulated through the dynamic digital twin of the spine.

[0042] Calculate the initial displacement field of the initial static equilibrium state;

[0043] Define the time step and dynamic load of the dynamic spinal digital twin;

[0044] The dynamic changes of the spine are simulated based on the time step and the dynamic load.

[0045] Based on the initial displacement field, analyze the displacement vector of the corresponding node of the spine in the dynamic change state;

[0046] The dynamic deformation field of the spine is determined based on the displacement vector.

[0047] Optionally, generating the planned motion path of the robotic arm corresponding to the spine includes:

[0048] Identify the workspace of the robotic arm;

[0049] The dynamic deformation field of the spine is mapped to the workspace to obtain the mapped deformation field;

[0050] Based on the mapped deformation field, the target position sequence and attitude sequence of the robotic arm are calculated;

[0051] The planned motion path of the robotic arm is generated based on the target position sequence and the posture sequence.

[0052] Optionally, calculating the path error between the real-time motion path and the planned motion path includes:

[0053] The real-time motion path and the planned motion path are respectively subjected to time-series discretization processing to obtain the real-time path sequence and the planned path sequence;

[0054] Construct the real-time path vector set of the real-time path sequence and the planned path vector set of the planned path sequence respectively;

[0055] Based on the real-time path vector set and the planned path vector set, calculate the position error, attitude error, and angle error of the real-time motion path and the planned motion path;

[0056] The path error between the real-time motion path and the planned motion path is determined based on the position error, the attitude error, and the angle error.

[0057] Optionally, calculating the static compensation amount of the robotic arm using the impedance model includes:

[0058] The external force on the robotic arm is calculated using the impedance model.

[0059] The static displacement of the robotic arm is calculated based on the external force on the robotic arm and the corresponding stiffness coefficient of the robotic arm.

[0060] Based on the static displacement, the static compensation amount of the robotic arm is determined.

[0061] This invention, through the construction of a dynamic digital twin of the target spine, can realistically reflect the geometric and mechanical behavior of the spine in a dynamic environment, achieving adaptive response to the dynamic environment and laying a data foundation for subsequent robotic arm path planning. Optionally, this invention, by generating a planned motion path for the robotic arm corresponding to the spine, can dynamically compensate for real-time deformations of the spine during surgery (such as minute displacements caused by breathing, heartbeat, and muscle contraction), avoiding positioning deviations caused by anatomical structure movement. This invention, by generating pose calibration commands for the robotic arm, can significantly reduce trajectory tracking errors, improve motion control accuracy, thereby enhancing the robustness of robotic arm calibration and ensuring motion stability and reliability. This invention, by determining the stiffness coefficient, can further improve the stability and reliability of the robotic arm. The displacement compensation of the robotic arm can correct path errors caused by inertia, friction, and external disturbances during high-speed or high-load movements in real time, ensuring that the robotic arm moves strictly according to the planned path. In this embodiment, when the path error is less than the second threshold and not less than a preset safety threshold, the pulse voltage required for the corresponding micro-motion platform of the robotic arm can be calculated to achieve micro-displacement adjustment of the robotic arm, ensuring that the robotic arm does not overshoot or oscillate when approaching the target position, thereby achieving nanometer-level positioning accuracy. Finally, in this embodiment, by executing micrometer-level control of the robotic arm according to the micrometer-level calibration command, the displacement of the end effector of the robotic arm can be precisely controlled to achieve micrometer-level accuracy at the target position, thereby avoiding damage to surrounding tissues and improving surgical safety. Therefore, this invention improves the control accuracy and safety of robotic arms in spinal surgery. Attached Figure Description

[0062] Figure 1 A functional block diagram of a precision control system for a spinal surgery robotic arm that integrates deep learning and dynamic calibration, provided in an embodiment of the present invention;

[0063] Figure 2A flowchart illustrating a method for precise control of a spinal surgery robotic arm that integrates deep learning and dynamic calibration, provided as an embodiment of the present invention;

[0064] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0067] In practice, the server-side equipment deployed in the precision control system for a spinal surgery robotic arm integrating deep learning and dynamic calibration may consist of one or more devices. This system can be implemented as a business instance, a virtual machine, or hardware devices. For example, it can be implemented as a business instance deployed on one or more devices in a cloud node. Simply put, this system can be understood as software deployed on a cloud node, providing precise control services for the spinal surgery robotic arm using deep learning and dynamic calibration to various user terminals. Alternatively, it can also be implemented as a virtual machine deployed on one or more devices in a cloud node, with application software installed to manage each user terminal. Alternatively, this precision control system for spinal surgery robotic arms that integrates deep learning and dynamic calibration can also be implemented as a server consisting of numerous identical or different types of hardware devices, with one or more hardware devices set up to provide precision control services for spinal surgery robotic arms that integrate deep learning and dynamic calibration to various user terminals.

[0068] In terms of implementation, the precision control system for spinal surgery robotic arms integrating deep learning and dynamic calibration is mutually adaptable to the user terminal. Specifically, if the system is implemented as an application installed on a cloud service platform, the user terminal acts as a client establishing a communication connection with that application; or if the system is implemented as a website, the user terminal acts as a webpage; or if the system is implemented as a cloud service platform, the user terminal acts as a mini-program within an instant messaging application.

[0069] Reference Figure 1 The diagram shown is a functional block diagram of a precision control system for a spinal surgery robotic arm that integrates deep learning and dynamic calibration, provided in an embodiment of the present invention.

[0070] The precision control system 100 for a spinal surgery robotic arm integrating deep learning and dynamic calibration described in this invention can be located on a cloud server. In terms of implementation, it can be used as one or more service devices, or as an application installed in the cloud (e.g., a server, server cluster, etc., for precision control of a spinal surgery robotic arm integrating deep learning and dynamic calibration), or it can be developed as a website. Depending on the functions implemented, the precision control system 100 for a spinal surgery robotic arm integrating deep learning and dynamic calibration includes a digital twin construction module 101, a motion path analysis module 102, a coarse-level control module 103, a fine-level control module 104, and a micrometer-level control module 105.

[0071] In this embodiment of the invention, in the tracking of precise control of the spinal surgery robotic arm integrating deep learning and dynamic calibration, each of the above-mentioned modules can be implemented independently and called upon other modules. This calling can be understood as one module connecting to multiple modules of another type and providing corresponding services to those connected modules. In the precise control system of the spinal surgery robotic arm integrating deep learning and dynamic calibration provided by this embodiment of the invention, the applicable scope of the precise control architecture of the spinal surgery robotic arm integrating deep learning and dynamic calibration can be adjusted by adding modules and directly calling them without modifying the program code, achieving cluster-style horizontal expansion to quickly and flexibly expand the system. In practical applications, the above modules can be set in the same device or different devices, or in virtual devices, such as service instances in a cloud server.

[0072] The following describes, with reference to specific embodiments, each component and specific workflow of the precision control system for a spinal surgery robotic arm that integrates deep learning and dynamic calibration.

[0073] The digital twin construction module 101 is used to acquire multimodal data of the target object, wherein the multimodal data includes CT image data, three-dimensional point cloud data and instrument contact force data, so as to construct a dynamic spinal digital twin of the target object.

[0074] This invention provides a data foundation for subsequent data analysis, path planning, and precise control by acquiring multimodal data of the target object. The multimodal data refers to a collection of data with different physical properties, information dimensions, or expression forms acquired through different types of sensors, devices, and acquisition methods. The CT image data refers to high-resolution three-dimensional medical image data acquired through computed tomography. The three-dimensional point cloud data provides real-time, dynamic spatial position and posture data of the spine during surgery. The instrument contact force data provides real-time information on the physical interaction between the robotic arm and spinal tissue, including the magnitude, direction, and torque of the contact force.

[0075] This invention, through the construction of a dynamic digital twin of the target spine, can realistically reflect the geometric and mechanical behavior of the spine in a dynamic environment, achieving adaptive response to the dynamic environment and laying a data foundation for subsequent robotic arm path planning. The dynamic digital twin of the spine refers to a virtual digital model constructed based on real patient spinal data, capable of real-time dynamic updates, and simulating the deformation and mechanical behavior of the spine during movement.

[0076] As an embodiment of the present invention, constructing a dynamic digital twin of the spine of the target object includes:

[0077] Construct an image coordinate system for the CT image data corresponding to the target object and a surgical three-dimensional coordinate system for the three-dimensional point cloud data of the CT image data corresponding to the target object, respectively;

[0078] Determine the transformation matrix between the image coordinate system and the surgical three-dimensional coordinate system;

[0079] Based on the transformation matrix, spatiotemporal registration is performed on the multimodal data corresponding to the target object to obtain multimodal registration data;

[0080] Multimodal features are extracted from the multimodal registration data to perform data fusion on the multimodal registration data, resulting in fused data.

[0081] Based on the fused data, a three-dimensional model of the spine of the target object is fitted;

[0082] The dynamic behavior of the three-dimensional spinal model is simulated to construct a dynamic digital twin of the spine.

[0083] The image coordinate system refers to the coordinate system used to describe the spatial position of pixels in a CT slice. The surgical 3D coordinate system refers to the coordinate system based on the intraoperative 3D point cloud acquisition device (such as a laser scanner). The transformation matrix refers to the matrix used to map points from the image coordinate system to the surgical coordinate system. The multimodal registration data refers to a unified dataset formed by spatiotemporally aligning data from different sources (CT, point cloud, force sensor). The multimodal features refer to representative information extracted from the registered multimodal data, such as geometric features (e.g., curvature, surface normal vector), texture features (e.g., grayscale distribution), and statistical features (e.g., mean, variance). The fused data refers to a dataset that integrates different modal features into a unified expression. The spinal 3D model refers to a 3D geometric model with realistic anatomical structural features reconstructed based on the fused data. The dynamic behavior state refers to the mechanical response and deformation state of the spinal 3D model under different loads, postures, and surgical operations, such as physical quantities like displacement, stress, strain, and rotation angle.

[0084] Optionally, the transformation matrix between the image coordinate system and the surgical three-dimensional coordinate system can be determined by the paired point method, such as singular value decomposition algorithm, iterative nearest point algorithm, etc.

[0085] Optionally, the fused data can be obtained through multimodal feature fusion techniques, such as weighted fusion, deep learning fusion, etc.

[0086] Optionally, the dynamic digital twin of the spine in the three-dimensional model of the spine can be constructed using deep learning techniques, such as convolutional neural networks, recurrent neural networks, generative adversarial networks, etc.

[0087] The motion path analysis module 102 is used to calculate the dynamic deformation field of the spine corresponding to the target object based on the dynamic digital twin of the spine, so as to generate the planned motion path of the robotic arm corresponding to the spine.

[0088] This invention, through the calculation of the dynamic deformation field of the spine corresponding to the target object based on the dynamic digital twin of the spine, can reflect in real time the minute deformations of the spine caused by respiration, muscle contraction, instrument operation, etc. during surgery. Based on this, the surgical path can be adjusted in real time to avoid misoperation caused by anatomical structural deviations. The dynamic deformation field refers to a mathematical expression describing the changes in the geometry and spatial position of the spine over time under the influence of external forces, movement, and physiological activities (such as respiration, heartbeat, and muscle contraction).

[0089] As an embodiment of the present invention, calculating the dynamic deformation field of the spine corresponding to the target object based on the dynamic digital twin of the spine includes:

[0090] The initial static equilibrium state of the spine is simulated through the dynamic digital twin of the spine.

[0091] Calculate the initial displacement field of the initial static equilibrium state;

[0092] Define the time step and dynamic load of the dynamic spinal digital twin;

[0093] The dynamic changes of the spine are simulated based on the time step and the dynamic load.

[0094] Based on the initial displacement field, analyze the displacement vector of the corresponding node of the spine in the dynamic change state;

[0095] The dynamic deformation field of the spine is determined based on the displacement vector.

[0096] The initial static equilibrium state refers to the state of the spine when there are no external forces acting on it (except gravity) and all tissues (vertebrae, ligaments, and muscles) are in mechanical equilibrium. The initial displacement field refers to the set of minute displacement vectors of each node of the spine relative to the three-dimensional model of the spine in the initial static equilibrium state. The time step refers to the length of a series of small time intervals discretized from continuous time in dynamic simulation. The dynamic load refers to the external force that varies with time and is applied to the spinal model to simulate dynamic disturbances during surgery. The dynamic change state refers to the constantly changing mechanical state of the spine under dynamic loads over time. The displacement vector refers to the change in position of a node in the spinal model relative to the initial static equilibrium state during the dynamic process.

[0097] Optionally, the initial displacement field of the initial static equilibrium state can be calculated using a machine learning model, such as random forest or GNN.

[0098] Optionally, the displacement vector of the corresponding node of the spine in the dynamic change state can be analyzed by deep learning techniques, such as long short-term memory networks, CNN-LSTM hybrid models, etc.

[0099] In another implementation, the dynamic load is expressed by the following formula:

[0100]

[0101] in, express Dynamic load at any given moment, Indicates the load amplitude. Represents the sine function. Represents pi (π). Indicates the load frequency. Indicates the time.

[0102] For example, there is a periodic force on the spine caused by respiration, with a load amplitude of 10N, a load frequency of 0.25Hz, and a duration of 4s. Similarly, , , .

[0103] This invention, through the generation of a planned motion path for the robotic arm corresponding to the spine, dynamically compensates for real-time deformations of the spine during surgery (such as minute displacements caused by breathing, heartbeat, and muscle contraction), thus avoiding positioning deviations caused by anatomical structure movement. The planned motion path refers to an optimal motion trajectory from the starting position to the target position designed for the surgical robotic arm based on the spatiotemporal characteristics of the spine in a dynamic deformation field.

[0104] As an embodiment of the present invention, generating the planned motion path of the robotic arm corresponding to the spine includes:

[0105] Identify the workspace of the robotic arm;

[0106] The dynamic deformation field of the spine is mapped to the workspace to obtain the mapped deformation field;

[0107] Based on the mapped deformation field, the target position sequence and attitude sequence of the robotic arm are calculated;

[0108] The planned motion path of the robotic arm is generated based on the target position sequence and the posture sequence.

[0109] The workspace refers to the set of all spatial points that the robotic arm can reach. The mapped deformation field refers to the deformation field transformed from the dynamic deformation field of the spine to the workspace coordinate system of the robotic arm. The target position sequence refers to the set of spatial position points that the robotic arm needs to reach sequentially during its movement. The posture sequence refers to the set of postures that the robotic arm needs to maintain sequentially during its movement.

[0110] Optionally, the target position sequence and attitude sequence of the robotic arm can be calculated using a deformation field interpolation algorithm, such as a radial basis function neural network.

[0111] Optionally, the planned motion path of the robotic arm corresponding to the spine can be generated by a path planning algorithm, such as a fast expanding random tree, a probabilistic route map, or an optimization algorithm.

[0112] The coarse-level control module 103 is used to acquire the real-time motion path of the robotic arm and calculate the path error between the real-time motion path and the planned motion path. When the path error is not less than a preset first threshold, the translation vector and rotation vector of the robotic arm are calculated to generate the pose calibration command of the robotic arm. According to the pose calibration command, the coarse-level control of the robotic arm is executed.

[0113] This invention enables precise tracking of the robotic arm's real-time motion path, avoiding surgical errors caused by deviations in the robotic arm's movement. The real-time motion path refers to the spatial motion trajectory recorded and fed back by the robotic arm during the surgical procedure.

[0114] This invention, through calculating the path error between the real-time motion path and the planned motion path, accurately assesses the deviation between the robotic arm's end effector and the predetermined path, providing a basis for subsequent error compensation and path calibration. The path error refers to the difference between the actual motion trajectory and the preset planned trajectory of the robotic arm during surgical tasks.

[0115] As an embodiment of the present invention, the calculation of the path error between the real-time motion path and the planned motion path includes:

[0116] The real-time motion path and the planned motion path are respectively subjected to time-series discretization processing to obtain the real-time path sequence and the planned path sequence;

[0117] Construct the real-time path vector set of the real-time path sequence and the planned path vector set of the planned path sequence respectively;

[0118] Based on the real-time path vector set and the planned path vector set, calculate the position error, attitude error, and angle error of the real-time motion path and the planned motion path;

[0119] The path error between the real-time motion path and the planned motion path is determined based on the position error, the attitude error, and the angle error.

[0120] The real-time path sequence refers to the ordered set of points formed after the real-time motion path has been discretized over time. The planned path sequence refers to the ordered set of points formed after the ideal motion trajectory of the robotic arm has been discretized in the time dimension. The real-time path vector set refers to the set of feature vectors extracted from the real-time path sequence for error calculation. The planned path vector set refers to the set of feature vectors extracted from the planned path sequence for error comparison. The position error refers to the Euclidean distance deviation between the position at a specific moment in the real-time path sequence and the corresponding position at a specific moment in the planned path sequence. The attitude error refers to the deviation between the attitude at a specific moment in the real-time path sequence and the attitude at a corresponding moment in the planned path sequence. The angle error refers to the angular deviation between the real-time path vector set and the planned path vector set.

[0121] Optionally, the real-time path sequence can be obtained using a dynamic time warping discretization method.

[0122] Optionally, the position error between the real-time motion path and the planned motion path can be calculated using the Euclidean distance method. The attitude error can be calculated using the quaternion difference method.

[0123] This invention, by calculating the translation and rotation vectors of the robotic arm when the path error is not less than a preset first threshold, can accurately quantify the position and orientation deviations of the robotic arm's end effector in space. This provides a precise mathematical basis for subsequent pose calibration, ensuring that the robotic arm can accurately adjust to the target position and posture. The preset first threshold refers to an error tolerance upper limit pre-set in the robotic arm control system, used to determine whether the deviation between the real-time motion path and the planned path of the robotic arm exceeds a safe range; in this application, it can be set to 0.5 mm. The translation vector describes the amount of positional change the robotic arm needs to move in three-dimensional space. The rotation vector is a mathematical tool used to represent the rotational changes the robotic arm needs to make in space.

[0124] As an embodiment of the present invention, when the path error is not less than a preset first threshold, calculating the translation vector and rotation vector of the robotic arm includes:

[0125] Based on the path error, the actual pose and planned pose of the robotic arm are determined;

[0126] Calculate the translation vector of the robotic arm based on the actual pose and the planned pose;

[0127] Determine the actual pose four elements of the actual pose and the planned pose four elements of the planned pose, respectively;

[0128] The relative rotation quaternion of the robotic arm is calculated based on the actual pose quaternion and the planned pose quaternion.

[0129] Based on the relative rotation quaternion, calculate the rotation angle and rotation unit vector of the robotic arm;

[0130] The rotation vector of the robotic arm is determined based on the rotation angle and the rotation unit vector.

[0131] The actual pose refers to the robot arm's true position and posture in space. The planned pose refers to the position and posture the robot arm is expected to achieve during motion planning. The actual pose quaternion refers to the quaternion describing the robot arm's actual posture. The planned pose quaternion refers to the quaternion describing the robot arm's desired posture. The relative rotation quaternion refers to the amount of rotation required to rotate from the actual posture to the planned posture. The rotation angle refers to the angle by which the robot arm needs to rotate around a certain axis. The rotation unit vector refers to the unit vector in the direction of the rotation axis.

[0132] Optionally, the translation vector of the robotic arm can be calculated using the direct interpolation method.

[0133] As another implementation, the relative rotation quaternion can be calculated using the following formula:

[0134]

[0135] in, Represents a relative rotation quaternion. This represents the four elements of the planned pose. The four elements representing the actual attitude. The four elements representing the actual attitude. This represents the quaternion multiplication operator.

[0136] In another implementation, the rotation angle and the rotation unit vector are calculated using the following formula:

[0137]

[0138]

[0139] in, Indicates the rotation angle. Represents the inverse cosine function. Represent the real part of a relative rotation quaternion. Indicates the imaginary part of the relative rotation quaternion. Components on the axis, Indicates the imaginary part of the relative rotation quaternion. Components on the axis, Indicates the imaginary part of the relative rotation quaternion. Components on the axis, This represents the sine function.

[0140] For example, there are four elements for planning pose. Actual attitude four elements Then relative rotation quaternion Then the rotation angle Rotation unit vector .

[0141] This invention, through the generation of pose calibration commands for the robotic arm, can significantly reduce trajectory tracking errors, improve motion control accuracy, and thus enhance the calibration robustness of the robotic arm, ensuring motion stability and reliability. Specifically, the pose calibration commands are control instructions generated based on the translation and rotation vectors of the robotic arm, used to dynamically adjust the robotic arm's pose.

[0142] Optionally, the pose calibration instructions for the robotic arm can be generated using a reinforcement learning algorithm.

[0143] In this embodiment of the invention, by executing coarse-level control of the robotic arm according to the pose calibration command, a larger step size and gain coefficient can be used, enabling the robotic arm to approach the target pose more quickly, thus allowing time for subsequent fine-tuning and improving overall control efficiency. The coarse-level control refers to a stage in the motion control of the robotic arm used to make preliminary adjustments to the robotic arm's pose (position and attitude) within a short period of time and with a large amplitude, enabling it to quickly approach the target pose.

[0144] The precision control module 104 is used to calculate the stiffness coefficient of the robotic arm when the path error is less than the first threshold and not less than the preset second threshold, determine the displacement compensation amount of the robotic arm based on the stiffness coefficient, generate an impedance calibration command for the robotic arm, and execute precision control of the robotic arm based on the impedance calibration command.

[0145] In this embodiment of the invention, when the path error is less than a first threshold and not less than a preset second threshold, the stiffness coefficient of the robotic arm is calculated, and the end-effector stiffness can be automatically adjusted based on bone density and real-time contact force to avoid instrument slippage caused by sudden changes in tissue characteristics, thereby improving surgical safety. The preset second threshold refers to the key dividing value that triggers different control modes of the robotic arm, and in this application, it can be set to 0.1 mm.

[0146] As an embodiment of the present invention, when the path error is less than the first threshold and not less than a preset second threshold, calculating the stiffness coefficient of the robotic arm includes:

[0147] Extract the CT grayscale values ​​and bone-device contact force from the multimodal data corresponding to the robotic arm;

[0148] Based on the CT grayscale values, analyze the current bone density of the spinal region corresponding to the robotic arm;

[0149] The stiffness coefficient of the robotic arm is calculated based on the current bone density in the region, the bone-device contact force, and the path error.

[0150] Wherein, the CT grayscale value refers to the relative density value of pixels in a CT scan. The bone-device contact force refers to the real-time interaction force between the robotic arm and bone tissue. The current region bone density refers to the bone tissue density of the current operating area. Healthy cortical bone refers to the densest part of a normal human skeleton. The cortical bone density threshold is a density boundary value used to distinguish healthy cortical bone from other bone tissues, which can be set to 1200 HU in this application. The bone density-stiffness nonlinear relationship coefficient is a coefficient describing the nonlinear relationship between bone density and stiffness. The force feedback gain coefficient is a weight used to adjust the influence of the bone-device contact force on the stiffness coefficient.

[0151] Optionally, the CT grayscale values ​​can be extracted using deep learning models, such as U-Net, VisionTransformer, etc.

[0152] Optionally, the bone density of the current region can be analyzed by establishing a calibration curve between CT grayscale values ​​and bone density.

[0153] As another implementation, the stiffness coefficient is calculated using the following formula:

[0154]

[0155] in, Indicates the stiffness coefficient. This indicates the basic stiffness of the spine corresponding to healthy cortical bone. Indicates the current bone density in the region. This indicates the cortical bone mineral density threshold corresponding to the spine. This represents the coefficient indicating the nonlinear relationship between bone mineral density and stiffness. Indicates the force feedback gain coefficient. Indicates bone-device contact force. This indicates path error.

[0156] It needs to be explained that in this application, the formula... This represents the coefficient indicating the nonlinear relationship between bone mineral density and stiffness, with a value of 1.5. This represents the force feedback gain coefficient, with a value of (0,1]. The larger the value, the more significant the effect of force feedback on stiffness.

[0157] For example, given a base stiffness of 500 N / mm, a bone-device contact force of 12 N, a path error of 0.4 mm, a bone density-stiffness nonlinearity coefficient of 1.5, a force feedback gain coefficient of 0.2, a current region bone density of 850 HU, and a cortical bone density threshold of 1200 HU, then the stiffness coefficient... N / mm.

[0158] This invention, by determining the displacement compensation amount of the robotic arm based on the stiffness coefficient, can correct path errors caused by the robotic arm's inertia, friction, and external disturbances during high-speed or high-load movements in real time, ensuring that the robotic arm moves strictly according to the planned path. The displacement compensation amount refers to the displacement vector that the robotic arm's end effector needs to adjust in real time.

[0159] As an embodiment of the present invention, determining the displacement compensation amount of the robotic arm based on the stiffness coefficient includes:

[0160] Based on the stiffness coefficient, analyze the damping coefficient of the robotic arm;

[0161] By combining the damping coefficient and the stiffness coefficient, an impedance model of the robotic arm is fitted.

[0162] Based on the path error corresponding to the robotic arm, the static compensation amount of the robotic arm is calculated using the impedance model.

[0163] Determine the respiratory motion pattern of the target object corresponding to the robotic arm;

[0164] Based on the breathing motion pattern and the static compensation amount, the displacement compensation amount of the robotic arm is analyzed.

[0165] The damping coefficient refers to a parameter describing the energy dissipation rate of the robotic arm system during movement. The impedance model is a mathematical model describing the dynamic response behavior of the robotic arm's end effector under external forces. The static compensation amount refers to the displacement adjustment amount calculated by the control algorithm to compensate for end-effector position deviations caused by factors such as gravity, static load, and structural deformation during low-speed robotic arm movement. The respiratory motion pattern refers to the periodic three-dimensional spatial displacement, deformation, and velocity changes of the spine during natural breathing.

[0166] Optionally, the impedance model of the robotic arm can be fitted using meta-learning methods such as MAML and Reptile.

[0167] Optionally, calculating the static compensation amount of the robotic arm using the impedance model includes:

[0168] The external force on the robotic arm is calculated using the impedance model.

[0169] The static displacement of the robotic arm is calculated based on the external force on the robotic arm and the corresponding stiffness coefficient of the robotic arm.

[0170] Based on the static displacement, the static compensation amount of the robotic arm is determined.

[0171] The external force on the robotic arm refers to the force exerted on the end effector of the robotic arm by the external environment and load, such as gravity load and contact force. The static displacement refers to the displacement generated when the end effector of the robotic arm reaches a steady state (i.e., both velocity and acceleration are zero) under the action of the external force. The static compensation amount refers to the additional position adjustment amount that needs to be applied to compensate for the static displacement generated by the robotic arm under the action of the external force.

[0172] As another implementation, the damping coefficient can be calculated using the following formula:

[0173]

[0174] in, Indicates the damping coefficient. Indicates the damping ratio. Indicates the stiffness coefficient. Indicates virtual quality.

[0175] It needs to be explained that in this application, the formula... This represents the damping ratio, with a value of 0.7. This represents the virtual mass, with a value of 0.1 kg.

[0176] This invention enables smooth switching between position control and force control by generating impedance calibration commands for the robotic arm, adapting to complex task requirements and avoiding damage caused by excessive rigidity or insufficient accuracy due to excessive flexibility. The impedance calibration commands refer to a set of control instructions generated in the robotic arm control system to adjust the robotic arm's impedance parameters so that the robotic arm's dynamic response characteristics (such as stiffness, damping, and inertia) meet specific task requirements.

[0177] This invention, through the implementation of the impedance calibration command, enables the robotic arm to generate minute displacements under external forces, avoiding rigid collisions and improving the safety and comfort of human-robot collaboration. The precise control refers to the high-precision, small-amplitude, and refined pose and force control adjustments performed on the robotic arm's end effector after initial coarse-level control in the robot control system.

[0178] The micron-level control module 105 is used to calculate the pulse voltage required by the micro-motion platform corresponding to the robotic arm when the path error is less than the second threshold and not less than the preset safety threshold, so as to generate the micron-level calibration command of the robotic arm and execute the micron-level control of the robotic arm according to the micron-level calibration command.

[0179] This invention achieves minute displacement adjustments of the robotic arm by calculating the pulse voltage required for the corresponding micro-motion platform when the path error is less than the second threshold and not less than a preset safety threshold. This ensures that the robotic arm does not overshoot or oscillate when approaching the target position, thereby achieving nanometer-level positioning accuracy. The preset safety threshold is a key error limit set in robotic arm motion control to ensure the safety, stability, and reliability of the system; in this application, it can be set to 0.01 mm.

[0180] As an embodiment of the present invention, when the path error is less than the second threshold and not less than a preset safety threshold, calculating the pulse voltage required by the micro-motion platform corresponding to the robotic arm includes:

[0181] Determine the mapping coefficients between the robotic arm's coordinate system and the micro-motion platform's coordinate system;

[0182] Based on the path error and the mapping coefficient, the amount of displacement that the robotic arm needs to compensate is determined;

[0183] The displacement-voltage linear relationship of the micro-motion platform is analyzed to determine the displacement-voltage conversion coefficient of the micro-motion platform;

[0184] The pulse voltage of the micro-motion platform is determined based on the displacement-voltage conversion coefficient and the displacement amount.

[0185] The mapping coefficient refers to the transformation relationship between the robotic arm coordinate system and the micro-motion platform coordinate system. The displacement refers to the displacement that the micro-motion platform needs to compensate for. The displacement-voltage linear relationship refers to an approximately linear functional relationship between the driving voltage input to the micro-motion platform and the mechanical displacement it generates. The displacement-voltage conversion coefficient refers to the displacement that can be driven per unit voltage.

[0186] Optionally, the mapping coefficients between the robotic arm's coordinate system and the micro-motion platform's coordinate system can be determined using augmented reality-assisted calibration methods, such as HoloLens and Magic Leap.

[0187] Optionally, the displacement-voltage linear relationship of the micro-motion platform can be analyzed using a recursive least squares algorithm.

[0188] This invention enables micron- or even nanometer-level displacement control by generating micron-level calibration commands for the robotic arm, thereby improving surgical precision and enhancing surgical quality and safety. Specifically, the micron-level calibration commands refer to control commands used to drive the robotic arm's end effector to perform micron-level (μm) or even sub-micron-level precision displacement or attitude adjustments.

[0189] This invention, through micron-level calibration instructions, enables precise control of the robotic arm's end effector displacement to achieve micron-level accuracy at the target position, thereby avoiding damage to surrounding tissues and improving surgical safety. The micron-level control refers to precise control of the robotic arm's end effector's displacement, velocity, force, or posture at micron or even sub-micron levels.

[0190] This invention, through the construction of a dynamic digital twin of the target spine, can realistically reflect the geometric and mechanical behavior of the spine in a dynamic environment, achieving adaptive response to the dynamic environment and laying a data foundation for subsequent robotic arm path planning. Optionally, this invention, by generating a planned motion path for the robotic arm corresponding to the spine, can dynamically compensate for real-time deformations of the spine during surgery (such as minute displacements caused by breathing, heartbeat, and muscle contraction), avoiding positioning deviations caused by anatomical structure movement. This invention, by generating pose calibration commands for the robotic arm, can significantly reduce trajectory tracking errors, improve motion control accuracy, thereby enhancing the robustness of robotic arm calibration and ensuring motion stability and reliability. This invention, by determining the stiffness coefficient, can further improve the stability and reliability of the robotic arm. The displacement compensation of the robotic arm can correct path errors caused by inertia, friction, and external disturbances during high-speed or high-load movements in real time, ensuring that the robotic arm moves strictly according to the planned path. In this embodiment, when the path error is less than a second threshold and not less than a preset safety threshold, the pulse voltage required for the corresponding micro-motion platform of the robotic arm can be calculated to achieve micro-displacement adjustment of the robotic arm, ensuring that the robotic arm does not overshoot or oscillate when approaching the target position, thereby achieving nanometer-level positioning accuracy. Finally, in this embodiment, by executing micrometer-level control of the robotic arm according to the micrometer-level calibration command, the displacement of the end effector of the robotic arm can be precisely controlled to achieve micrometer-level accuracy at the target position, thereby avoiding damage to surrounding tissues and improving surgical safety. Therefore, this invention can improve the control accuracy and safety of robotic arms in spinal surgery.

[0191] like Figure 2 The diagram shown is a flowchart illustrating a method for precise control of a spinal surgery robotic arm integrating deep learning and dynamic calibration, according to an embodiment of the present invention. In this embodiment, the method for precise control of a spinal surgery robotic arm integrating deep learning and dynamic calibration includes:

[0192] S1. Acquire multimodal data of the target object, wherein the multimodal data includes CT image data, three-dimensional point cloud data and instrument contact force data, in order to construct a dynamic digital twin of the spine of the target object;

[0193] S2. Based on the dynamic digital twin of the spine, calculate the dynamic deformation field of the spine corresponding to the target object to generate the planned motion path of the robotic arm corresponding to the spine.

[0194] S3. Obtain the real-time motion path of the robotic arm and calculate the path error between the real-time motion path and the planned motion path. When the path error is not less than a preset first threshold, calculate the translation vector and rotation vector of the robotic arm to generate the pose calibration command of the robotic arm. According to the pose calibration command, perform coarse control of the robotic arm.

[0195] S4. When the path error is less than the first threshold and not less than the preset second threshold, calculate the stiffness coefficient of the robotic arm, determine the displacement compensation amount of the robotic arm based on the stiffness coefficient, generate the impedance calibration command of the robotic arm, and perform fine-tuning of the robotic arm based on the impedance calibration command.

[0196] S5. When the path error is less than the second threshold and not less than the preset safety threshold, calculate the pulse voltage required by the micro-motion platform corresponding to the robotic arm to generate a micron-level calibration command for the robotic arm, and perform micron-level control of the robotic arm according to the micron-level calibration command.

[0197] In the several embodiments provided by this invention, it should be understood that the provided systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0198] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0199] Finally, it should be noted that in the above embodiments, each embodiment can be combined with each other or independent. Deleting any one of them will not affect the technical implementation of other embodiments. The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A precision control system for a spinal surgery robotic arm integrating deep learning and dynamic calibration, characterized in that, The precision control system for spinal surgery robotic arms that integrates deep learning and dynamic calibration includes: A digital twin construction module is used to acquire multimodal data of a target object, wherein the multimodal data includes CT image data, three-dimensional point cloud data, and instrument contact force data, in order to construct a dynamic spinal digital twin of the target object; The motion path analysis module is used to calculate the dynamic deformation field of the spine corresponding to the target object based on the dynamic digital twin of the spine, so as to generate the planned motion path of the robotic arm corresponding to the spine. The coarse-level control module is used to acquire the real-time motion path of the robotic arm and calculate the path error between the real-time motion path and the planned motion path. When the path error is not less than a preset first threshold, the module calculates the translation vector and rotation vector of the robotic arm to generate the pose calibration command of the robotic arm. Based on the pose calibration command, the module executes the coarse-level control of the robotic arm. A precision control module is used to calculate the stiffness coefficient of the robotic arm when the path error is less than a first threshold and not less than a preset second threshold, determine the displacement compensation amount of the robotic arm based on the stiffness coefficient, generate an impedance calibration command for the robotic arm, and execute precision control of the robotic arm based on the impedance calibration command. The step of calculating the stiffness coefficient of the robotic arm when the path error is less than the first threshold and not less than the preset second threshold includes: extracting the CT grayscale value and bone-instrument contact force of the multimodal data corresponding to the robotic arm; analyzing the current regional bone density of the spinal region corresponding to the robotic arm based on the CT grayscale value; calculating the stiffness coefficient of the robotic arm based on the current regional bone density, the bone-instrument contact force, and the path error. Determining the displacement compensation amount of the robotic arm using the stiffness coefficient includes: analyzing the damping coefficient of the robotic arm based on the stiffness coefficient; fitting the impedance model of the robotic arm by combining the damping coefficient and the stiffness coefficient; calculating the static compensation amount of the robotic arm using the impedance model based on the path error corresponding to the robotic arm; determining the breathing motion law of the target object corresponding to the robotic arm; and analyzing the displacement compensation amount of the robotic arm based on the breathing motion law and the static compensation amount. The step of calculating the static compensation amount of the robotic arm using the impedance model includes: calculating the external force on the robotic arm using the impedance model; calculating the static displacement of the robotic arm based on the external force and the corresponding stiffness coefficient; and determining the static compensation amount of the robotic arm based on the static displacement. A micrometer-level control module is used to calculate the pulse voltage required by the micro-motion platform corresponding to the robotic arm when the path error is less than the second threshold and not less than a preset safety threshold, so as to generate a micrometer-level calibration command for the robotic arm, and execute micrometer-level control of the robotic arm according to the micrometer-level calibration command. The step of calculating the pulse voltage required by the micro-motion platform corresponding to the robotic arm when the path error is less than the second threshold and not less than the preset safety threshold includes: determining the mapping coefficient between the robotic arm coordinate system and the micro-motion platform coordinate system; determining the displacement amount that the robotic arm needs to compensate for based on the path error and the mapping coefficient; analyzing the displacement-voltage linear relationship of the micro-motion platform to determine the displacement-voltage conversion coefficient of the micro-motion platform; and determining the pulse voltage of the micro-motion platform based on the displacement-voltage conversion coefficient and the displacement amount.

2. The precision control system for a spinal surgery robotic arm integrating deep learning and dynamic calibration as described in claim 1, characterized in that, When the path error is not less than a preset first threshold, the translation vector and rotation vector of the robotic arm are calculated, including: Based on the path error, the actual pose and planned pose of the robotic arm are determined; Calculate the translation vector of the robotic arm based on the actual pose and the planned pose; Determine the actual pose four elements of the actual pose and the planned pose four elements of the planned pose, respectively; The relative rotation quaternion of the robotic arm is calculated based on the actual pose quaternion and the planned pose quaternion. Based on the relative rotation quaternion, calculate the rotation angle and rotation unit vector of the robotic arm; The rotation vector of the robotic arm is determined based on the rotation angle and the rotation unit vector.

3. The precision control system for a spinal surgery robotic arm integrating deep learning and dynamic calibration as described in claim 1, characterized in that, The construction of the dynamic spinal digital twin of the target object includes: Construct an image coordinate system for the CT image data corresponding to the target object and a surgical three-dimensional coordinate system for the three-dimensional point cloud data of the CT image data corresponding to the target object, respectively; Determine the transformation matrix between the image coordinate system and the surgical three-dimensional coordinate system; Based on the transformation matrix, spatiotemporal registration is performed on the multimodal data corresponding to the target object to obtain multimodal registration data; Multimodal features are extracted from the multimodal registration data to perform data fusion on the multimodal registration data, resulting in fused data. Based on the fused data, a three-dimensional model of the spine of the target object is fitted; The dynamic behavior of the three-dimensional spinal model is simulated to construct a dynamic digital twin of the spine.

4. The precision control system for a spinal surgery robotic arm integrating deep learning and dynamic calibration as described in claim 1, characterized in that, The step of calculating the dynamic deformation field of the spine corresponding to the target object based on the dynamic digital twin of the spine includes: The initial static equilibrium state of the spine is simulated through the dynamic digital twin of the spine. Calculate the initial displacement field of the initial static equilibrium state; Define the time step and dynamic load of the dynamic spinal digital twin; The dynamic changes of the spine are simulated based on the time step and the dynamic load. Based on the initial displacement field, analyze the displacement vector of the corresponding node of the spine in the dynamic change state; The dynamic deformation field of the spine is determined based on the displacement vector.

5. The precision control system for a spinal surgery robotic arm integrating deep learning and dynamic calibration as described in claim 1, characterized in that, The generation of the planned motion path for the robotic arm corresponding to the spine includes: Identify the workspace of the robotic arm; The dynamic deformation field of the spine is mapped to the workspace to obtain the mapped deformation field; Based on the mapped deformation field, the target position sequence and attitude sequence of the robotic arm are calculated; The planned motion path of the robotic arm is generated based on the target position sequence and the posture sequence.

6. The precision control system for a spinal surgery robotic arm integrating deep learning and dynamic calibration as described in claim 1, characterized in that, The calculation of the path error between the real-time motion path and the planned motion path includes: The real-time motion path and the planned motion path are respectively subjected to time-series discretization processing to obtain the real-time path sequence and the planned path sequence; Construct the real-time path vector set of the real-time path sequence and the planned path vector set of the planned path sequence respectively; Based on the real-time path vector set and the planned path vector set, calculate the position error, attitude error, and angle error of the real-time motion path and the planned motion path; The path error between the real-time motion path and the planned motion path is determined based on the position error, the attitude error, and the angle error.

Images