Master-slave operated puncture system and planning method
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU MICROPORT ORTHOBOT CO LTD
- Filing Date
- 2023-03-27
- Publication Date
- 2026-06-19
Smart Images
Figure CN116269812B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of robot navigation technology, and in particular to a master-slave puncture system and planning method. Background Technology
[0002] A biopsy is a routine medical examination in which a sample of tissue is extracted from the body of a target individual for examination. Before a biopsy, professionals usually plan the puncture path to ensure that the puncture can safely and accurately reach the abnormal point while avoiding important tissues in the target individual's body.
[0003] Existing puncture robots determine the optimal initial puncture position and orientation of the puncture needle based on pre-procedure medical images and intraoperative visual navigation registration. The robotic arm is then guided to the target position for needle implantation. However, the operator cannot visually and intuitively evaluate the needle insertion path based on the current operational scenario before implantation, nor can they visually and dynamically adjust the needle insertion position and puncture depth before the procedure, resulting in poor operational intuitiveness. Summary of the Invention
[0004] Therefore, it is necessary to provide a puncture system and planning method that can provide an intuitive evaluation of the needle insertion path based on the current operating scenario and a master-slave operation that can visualize and dynamically adjust the needle insertion position and puncture depth to address the above-mentioned technical problems.
[0005] Firstly, this application provides a master-slave puncture system, which includes:
[0006] The first robotic arm and the operating tool assembled at the end of the first robotic arm;
[0007] The second robotic arm is used to perform pre-operations. The second robotic arm includes a simulation model. The simulation model and the second robotic arm have a master-slave mapping relationship. The end of the simulation model is used to load the simulation device model.
[0008] A display device for displaying an image of a target object, and for overlaying a simulation device model onto the image of the target object; and
[0009] The control device is communicatively connected to the first robotic arm, the second robotic arm, and the display device. The control device controls the second robotic arm to perform pre-operations according to the target object, so that the simulation instrument model at the end of the simulation model moves to obtain the optimal target puncture position and target puncture posture relative to the target object. Based on the target puncture position and target puncture posture, the control device controls the first robotic arm to drive the operating tool to position itself at the target puncture position and target puncture posture.
[0010] In one embodiment, the control device controlling the second robotic arm to perform pre-operations according to the target object further includes causing the simulation model to move carrying the simulation instrument model to obtain the target puncture depth corresponding to the simulation instrument model.
[0011] In one embodiment, obtaining the target puncture depth corresponding to the simulation instrument model further includes: constraining the movement direction of the end of the second robotic arm to be the same as the axial direction of the simulation instrument model.
[0012] In one embodiment, a vision sensor is also included for receiving the real-time pose of the operating tool;
[0013] The control device includes a first control unit and a second control unit that are connected in communication. The first control unit is connected in communication with the first robotic arm and a vision sensor; the second control unit is connected in communication with the second robotic arm and a display device.
[0014] The second control unit obtains and sends the target puncture position and target puncture posture to the first control unit. The first control unit guides the first robotic arm to position the operating tool to the target puncture position and target puncture posture based on the target puncture posture, target puncture position and real-time pose.
[0015] In one embodiment, the joint configuration of the first robotic arm is the same as that of the second robotic arm; the second control unit is also used to acquire the joint control parameters of the second robotic arm under the target puncture position and the target puncture posture.
[0016] The first robotic arm and the second robotic arm are aligned by a calibration relationship. Based on the calibration relationship and joint control parameters, the joints of the first robotic arm are controlled so that the operating tool at the end of the first robotic arm is positioned at the target puncture position and the target puncture posture.
[0017] In one embodiment, the control device controls the second robotic arm to perform a pre-operation based on the target object, and the posture of the simulation device model loaded at the end of the simulation model remains fixed relative to the end of the simulation model.
[0018] In one embodiment, during the pre-operation of the second robotic arm controlled by the control device according to the target object, the control device constrains at least one parameter among the puncture position, puncture posture, and puncture depth, and controls the second robotic arm to perform the pre-operation, so that the simulated instrument model moves under the remaining parameters of puncture position, puncture posture, and puncture depth.
[0019] In one embodiment, the puncture location includes an initial puncture location and an end puncture location. The control device is used to constrain the initial puncture location, move the simulation instrument model to obtain multiple end puncture locations, and plan multiple first puncture paths based on the initial puncture location and the multiple end puncture locations. The multiple first puncture paths are displayed on the target object image. Alternatively, the control device is used to constrain the end puncture location, move the simulation instrument model to obtain multiple initial puncture locations, and plan multiple second puncture paths based on the end puncture location and the multiple initial puncture locations. The multiple second puncture paths are displayed on the target object image.
[0020] In one embodiment, the display device includes a projection unit that is communicatively connected to the control device. The projection unit acquires and projects at least the simulation device model and the target object image onto the target object body.
[0021] Secondly, this application also provides a planning method for master-slave operations, the method comprising:
[0022] Based on the target object, the second robotic arm performs pre-operations, causing the end effector of the second robotic arm's simulation model to move, carrying the simulation instrument model, to obtain the optimal target puncture position and posture relative to the target object; the simulation model and the second robotic arm have a master-slave mapping relationship; and
[0023] Based on the target puncture location and target puncture posture, the first robotic arm is controlled to move the operating tool to the target puncture location and target puncture posture.
[0024] In one embodiment, the second robotic arm is controlled to perform a pre-operation based on the target object, causing the end effector of the second robotic arm carrying the simulated instrument model to move, including:
[0025] Constrain at least one of the parameters of puncture position, puncture posture, and puncture depth, and control the second robotic arm to perform a pre-operation so that the simulated instrument model moves under the remaining parameters of puncture position, puncture posture, and puncture depth.
[0026] In one embodiment, the puncture location includes an initial puncture location and a final puncture location, and controlling the second robotic arm to perform pre-operations includes:
[0027] The initial puncture position is constrained, and the moving simulation instrument model obtains multiple puncture endpoint positions. Multiple first puncture paths are planned based on the initial puncture position and multiple puncture endpoint positions. The endpoint positions are constrained, and the moving simulation instrument model obtains multiple puncture initial positions. Multiple second puncture paths are planned based on the endpoint positions and multiple puncture initial positions.
[0028] In one embodiment, the method further includes:
[0029] Obtain the image of the target object;
[0030] Overlaying a simulated instrument model onto an image of the target object; and
[0031] Display the simulation device model and the target object image onto the target object itself.
[0032] Thirdly, this application also provides a master-slave puncture device. The device includes:
[0033] The master-slave operation module is used to control the second robotic arm to perform pre-operations based on the target object, causing the end effector of the second robotic arm's simulation model to move, carrying the simulation instrument model, to obtain the optimal target puncture position and target puncture posture relative to the target object; the simulation model and the second robotic arm have a master-slave mapping relationship; and
[0034] The execution module is used to control the first robotic arm to position the operating tool to the target puncture position and target puncture posture based on the target puncture position and target puncture posture.
[0035] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:
[0036] Based on the target object, the second robotic arm performs pre-operations, causing the end effector of the second robotic arm's simulation model to move, carrying the simulation instrument model, to obtain the optimal target puncture position and posture relative to the target object; the simulation model and the second robotic arm have a master-slave mapping relationship; and
[0037] Based on the target puncture location and target puncture posture, the first robotic arm is controlled to move the operating tool to the target puncture location and target puncture posture.
[0038] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:
[0039] Based on the target object, the second robotic arm performs pre-operations, causing the end effector of the second robotic arm's simulation model to move, carrying the simulation instrument model, to obtain the optimal target puncture position and posture relative to the target object; the simulation model and the second robotic arm have a master-slave mapping relationship; and
[0040] Based on the target puncture location and target puncture posture, the first robotic arm is controlled to move the operating tool to the target puncture location and target puncture posture.
[0041] Sixthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:
[0042] Based on the target object, the second robotic arm performs pre-operations, causing the end effector of the second robotic arm's simulation model to move, carrying the simulation instrument model, to obtain the optimal target puncture position and posture relative to the target object; the simulation model and the second robotic arm have a master-slave mapping relationship; and
[0043] Based on the target puncture location and target puncture posture, the first robotic arm is controlled to move the operating tool to the target puncture location and target puncture posture.
[0044] The aforementioned master-slave puncture system and planning method, by setting up a second robotic arm, includes a simulation model. The simulation model and the second robotic arm have a master-slave mapping relationship. The end effector of the simulation model is used to load a simulated instrument model. The system controls the second robotic arm to perform pre-operations, causing the simulated instrument model at the end effector to move to obtain the optimal target puncture position and posture relative to the target object. In this system, the operator can visually and intuitively assess the feasibility of the puncture position and posture based on the current real-world operating scenario before the operation. Furthermore, the visual master-slave teleoperation dynamically adjusts the puncture posture, puncture position, and puncture depth to determine the optimal target puncture position and posture. Then, it guides the first robotic arm to move to the optimal target puncture position and posture, making the doctor's operation guidance more intuitive. Attached Figure Description
[0045] Figure 1 This is a diagram illustrating the application environment of a master-slave puncture system in one embodiment.
[0046] Figure 2 This is a master-slave control flowchart for a simulation device model in one embodiment;
[0047] Figure 3 A flowchart of the master-slave mapping control algorithm in one embodiment; a schematic diagram of the initial puncture position and puncture posture pointing to the scene in the simulation model;
[0048] Figure 4 This is a simulation device model control mode switching process in one embodiment;
[0049] Figure 5 This is a diagram illustrating the application environment of a visual sensor in a puncture system in one embodiment;
[0050] Figure 6 This is a schematic diagram of the puncture position and puncture posture of a simulation instrument model in one embodiment;
[0051] Figure 7This is a flowchart of the medical image and visual image fusion process in one embodiment;
[0052] Figure 8 This is a schematic diagram of image fusion between medical images and visual images during the puncture procedure in one embodiment;
[0053] Figure 9 This is a schematic diagram illustrating the determination of the target puncture depth in a simulated instrument model puncture scenario in one embodiment.
[0054] Figure 10 This is a flowchart of the axial control strategy for the second robotic arm in one embodiment;
[0055] Figure 11 This is a schematic diagram illustrating the determination of the optimal target puncture location in a puncture biopsy scenario, as shown in one embodiment.
[0056] Figure 12 This is a flowchart illustrating the control strategy for determining the optimal target puncture location in a multi-point puncture biopsy scenario in one embodiment.
[0057] Figure 13 This is a schematic diagram illustrating the determination of the optimal initial puncture position and the optimal puncture posture in a simulated instrument model puncture scenario in one embodiment.
[0058] Figure 14 A flowchart illustrating the control strategy for keeping the end-effector target position unchanged in one embodiment;
[0059] Figure 15 This is a schematic diagram illustrating the safety alarm prompts for the operation of a simulation device model in one embodiment;
[0060] Figure 16 This is a flowchart of a master-slave operation planning method in one embodiment;
[0061] Figure 17 This is a structural block diagram of a master-slave puncture device in one embodiment. Detailed Implementation
[0062] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0063] Currently, puncture paths are planned based on medical images, and then a robotic arm executes the procedure to the target location. However, before the actual needle insertion, it's impossible to visually adjust the appropriate puncture position and posture. Therefore, to address these issues, this application provides a master-slave puncture system that can be applied to applications such as... Figure 1The application environment shown includes a first robotic arm 2, an operating tool 3, a second robotic arm 8, a display device (not shown), and a control device (not shown).
[0064] The first robotic arm 2 is mounted on the robotic arm trolley 1, and the end of the first robotic arm 2 is equipped with an operating tool 3; the operating tool 3 may be a puncture guide, bone needle or other tools.
[0065] The second robotic arm 8 is used to perform pre-operations. The second robotic arm 8 includes a simulation model. The simulation model and the second robotic arm 8 have a master-slave mapping relationship. The end of the simulation model is used to load the simulation device model 9.
[0066] Pre-operation refers to the adjustments required by the second robotic arm 8 when controlling the movement of the simulation instrument model 9 according to the master-slave mapping relationship. For example, controlling the second robotic arm 8 to translate or rotate along a horizontal line. The simulation model is the simulated structure of the second robotic arm 8 in the simulation environment, and the simulation instrument model 9 is the simulated structure of the surgical tool in the simulation environment. The end effector loading of the simulation instrument model 9 means that the end effector of the simulation model of the second robotic arm 8 grips the simulation instrument model 9, and there is a fixed pose relationship between the gripping part and the simulation instrument model 9.
[0067] In this embodiment, a master-slave control scheme is used to control the movement of the simulation instrument model 9. Specifically, the second robotic arm 8 is the master operation end, and the simulation model is the slave operation end. The operator performs pre-operations through the second robotic arm 8 to make the simulation model carry the simulation instrument model 9 to move, thereby controlling the puncture position and puncture posture of the simulation instrument model 9, and can adjust the puncture position and puncture posture of the simulation instrument model 9 in real time and visually.
[0068] In some embodiments, the second robotic arm 8 includes three posture joints and three position joints. The axes of the three posture joints intersect at the end of the operating handle of the second robotic arm 8, that is, the movement of the posture joints of the second robotic arm 8 does not affect the end position of the second robotic arm 8.
[0069] In some embodiments, by fusing the coordinate system of the simulation model with the coordinate system of the second robotic arm 8, the Cartesian pose and joint positions of the simulation model can be calculated. In actual control, such as... Figure 2 As shown, the second robotic arm 8 includes redundant sensors. After processing by the signal input module and the master-end forward kinematics algorithm, the master-end Cartesian pose is calculated. Based on the master-slave mapping control algorithm and the master-end Cartesian pose, the Cartesian pose of the simulation device model 9 is obtained. Then, through the virtual robotic arm inverse kinematics, the command position of the virtual joint in the simulation model is calculated and controlled.
[0070] In some embodiments, the master-slave control strategy of this embodiment is as follows: relative to an initial reference benchmark, the relative motion displacement of the simulation device model 9 and the second robotic arm 8 is consistent, and their absolute attitude orientation remains consistent. For example... Figure 3 As shown, at the start of the master-slave operation, the initial position P0_init of the simulation model can be set to [0,0,0] by default. Based on the master end Cartesian relative displacement ΔP, the attitude input R, and the initial position P0_init of the simulation model, the master-slave mapping control algorithm calculates the desired end pose [R, P0_init+ΔP] of the simulation device model 9. Then, through the inverse kinematics of the robotic arm, the joint command position of the simulation model is solved to obtain the command position of the virtual joint. The simulation model is then controlled by the virtual joint control unit, thereby controlling the simulation model to move the simulation device model 9.
[0071] The display device is used to display the image of the target object and to overlay the simulation device model 9 onto the image of the target object.
[0072] The simulation model may or may not be displayed on a display device. The target object image refers to the medical image corresponding to the organism to be punctured, such as medical images of biological structures like the spine and lungs. The medical image can be a CT image or MRI image before the puncture.
[0073] The control device is communicatively connected to the first robotic arm 2, the second robotic arm 8, and the display device. The control device controls the second robotic arm 8 to perform pre-operations according to the target object, so that the simulation instrument model 9 at the end of the simulation model moves to obtain the optimal target puncture position 6 and target puncture posture 7 relative to the target object. Based on the target puncture position 6 and target puncture posture 7, the control device controls the first robotic arm 2 to drive the operating tool 3 to position itself at the target puncture position 6 and target puncture posture 7.
[0074] In the master-slave control of simulation device model 9, such as Figure 4 As shown, the motion dimensions of the simulation instrument model 9 mainly include three dimensions: puncture position, puncture posture, and puncture depth. When the control device remotely controls the simulation instrument model 9 to obtain the optimal target puncture position 6 and target puncture posture 7 relative to the target object, it can, depending on the application scenario requirements, lock one or two of the motion dimensions while independently controlling the motion of the other two or one motion dimension. Once the corresponding motion dimension is locked, during the master-slave operation of the simulation instrument model 9, the motion in that direction will be restricted to remain stationary, while other motion dimensions will adapt to the motion. The master-slave control strategy and the master-end motion control will also adapt accordingly, and the end pose of the simulation model will be remapped according to the master-slave control strategy.
[0075] It should be noted that during the pre-operation of the second robotic arm 8 controlled by the control device according to the target object, the posture of the simulation instrument model 9 loaded at the end of the simulation model remains fixed relative to the end of the simulation model, that is, the simulation instrument model 9 moves with the simulation model.
[0076] In some embodiments, when switching the control mode of the simulation device model 9, the control mode can be selected and the motion dimension can be locked / released through the navigation interface of the display device, or the operation can be selected based on the function selection button of the robotic arm.
[0077] In this embodiment, the selection criteria for the optimal target puncture position 6 and target puncture posture 7 are determined based on the implantation status of the simulation device model 9 and the target object. For example, if the simulation device model 9 penetrates the normal tissue of the target object, the current puncture position and current puncture posture of the simulation device model 9 are determined to be infeasible; if the simulation device model 9 does not penetrate the normal tissue of the target object, the current puncture position and current puncture posture of the simulation device model 9 are determined to be feasible, and the current puncture position and current puncture posture of the simulation device model 9 are taken as the target puncture position 6 and target puncture posture 7.
[0078] In the aforementioned master-slave puncture system, a second robotic arm 8 is incorporated, comprising a simulation model. The simulation model and the second robotic arm 8 are in a master-slave mapping relationship. The end of the simulation model is used to load a simulation instrument model 9. The system controls the second robotic arm 8 to perform pre-operations, causing the simulation instrument model 9 at the end of the simulation model to move to obtain the optimal target puncture position 6 and target puncture posture 7 relative to the target object. In this system, the operator can visually and intuitively assess the feasibility of the puncture position and posture based on the current real-world operating scenario before the operation. Furthermore, the visual master-slave teleoperation dynamically adjusts the puncture posture, puncture position, and puncture depth to determine the optimal target puncture position 6 and target puncture posture 7. Then, the first robotic arm 2 is guided to move to the optimal target puncture position 6 and target puncture posture 7, making the doctor's operation guidance more intuitive.
[0079] In one embodiment, such as Figure 5 As shown, the puncture system also includes a vision sensor 5, which is mounted on the navigation trolley 4 and is used to receive the real-time pose of the operating tool 3. The control device includes a first control unit and a second control unit that are connected in communication. The first control unit is connected in communication with the first robotic arm 2 and the vision sensor; the second control unit is connected in communication with the second robotic arm 8 and the display device. The second control unit obtains and sends the target puncture position 6 and the target puncture posture 7 to the first control unit. The first control unit guides the first robotic arm 2 to position the operating tool 3 to the target puncture position 6 and the target puncture posture 7 based on the target puncture posture 7, the target puncture position 6, and the real-time pose.
[0080] The schematic diagram of the first robotic arm 2 driving the operating tool 3 to position the target puncture position 6 and the target puncture posture 7 is shown below. Figure 6 As shown, the first robotic arm 2 drives the operating tool 3 to position it at the target puncture position 6, and implants it into the target object along the needle insertion direction corresponding to the target puncture posture 7.
[0081] In one embodiment, the joint configuration of the first robotic arm 2 is the same as that of the second robotic arm 8. The second control unit is also used to acquire the joint control parameters of the second robotic arm 8 at the target puncture position 6 and the target puncture posture 7. There is a calibration relationship between the first robotic arm 2 and the second robotic arm 8. According to the calibration relationship and the joint control parameters, each joint of the first robotic arm 2 is controlled so that the operating tool 3 at the end of the first robotic arm 2 is positioned at the target puncture position 6 and the target puncture posture 7.
[0082] The joint configurations of the first robotic arm 2 and the second robotic arm 8 are identical. This is so that after obtaining the target puncture position 6 and the target puncture posture 7, the joint control parameters of the first robotic arm 2 can be directly determined based on the joint control parameters and calibration relationship of the second robotic arm 8 at the target puncture position 6 and the target puncture posture 7. This eliminates the need to solve for the joint control parameters of the first robotic arm 2 again through inverse kinematics, reducing the computational load and improving the control accuracy of the first robotic arm 2. The calibration relationship between the first robotic arm 2 and the second robotic arm 8 is to ensure that the movements of the first robotic arm 2 and the second robotic arm 8 are consistent, improving the point accuracy and trajectory accuracy of the first robotic arm 2.
[0083] In this embodiment, the first control unit receives the real-time pose of the operating tool 3 through a vision sensor, and guides the first robotic arm 2 to position the operating tool 3 to the target puncture position 6 and the target puncture pose 7 based on the real-time pose, the target puncture posture 7, the target puncture position 6, and the real-time pose. This allows for precise control of the pose of the first robotic arm 2. Furthermore, by setting the joint configuration of the first robotic arm 2 and the second robotic arm 8 to be the same, the joint control parameters of the first robotic arm 2 can be directly determined based on the joint control parameters and calibration relationship of the second robotic arm 8 at the target puncture position 6 and the target puncture pose 7. This eliminates the need to solve for the joint control parameters of the first robotic arm 2 again through robot inverse kinematics, reducing the computational load and improving the control accuracy of the first robotic arm 2.
[0084] In one embodiment, to ensure that the target puncture posture 7, target puncture position 6, and real-time pose of the receiving manipulation tool 3 of the simulation instrument model 9 are in the same coordinate system, the control device of this embodiment fuses the medical image of the target object before puncture and the visual image during puncture, achieving registration from the three-dimensional image space to the navigation space, obtaining a virtual operation space image, and fusing the visual image during puncture with the coordinate system of the first robotic arm, which can calculate the three-dimensional spatial position of the simulation instrument model 9 in the virtual operation space image in real time. The virtual operation space image includes the target object image, at least one virtual puncture path, and the simulation instrument model 9; the virtual puncture path is determined based on the initial puncture position, the final puncture position, and the puncture posture of the simulation instrument model 9.
[0085] In some embodiments, multiple target markers are marked on the surface of the target object. The fusion process of pre-puncture medical images and visual images during the puncture procedure specifically includes the following steps:
[0086] Step 1: Determine the first spatial position of the target marker in the medical image and track the second spatial position of the target marker in the visual image in real time. Register the first spatial position and the second spatial position to obtain the first mapping relationship between the medical image coordinate system and the visual image coordinate system.
[0087] Optionally, multiple target marker points are marked on the surface of the target object, such as... Figure 7 As shown, before the puncture, the target object's marker points are scanned simultaneously to obtain medical images including the marker points. The navigation trolley performs 3D modeling based on the pre-operation medical images and determines the first spatial position of the target marker points in the 3D medical images. During the puncture, the visual sensor tracks the second spatial position of the target marker points in the visual images in real time, adapting the spatial positions of the target marker points in the medical images and visual images to complete the registration and fusion of the medical image coordinate system and the visual image coordinate system, thus achieving the fusion of medical images and visual images. A schematic diagram of the target marker points in the medical images is shown below. Figure 8 As shown.
[0088] In some embodiments, the target marker includes multiple marker metal balls, which are fixed on the surface of the target object. During the puncture, a registration algorithm is used to track the spatial position of the marker metal balls in the visual image in real time, and then registers them with the coordinates of the marker metal balls in the medical image to achieve the fusion of the medical image and the visual image.
[0089] Step 2: Based on the motion information of each joint of the first robotic arm 2, determine the end-effector pose of the first robotic arm 2 in the robot coordinate system, track the end-effector target point position of the first robotic arm 2 in real time in the visual image, register the end-effector target point position with the end-effector pose of the first robotic arm 2, and obtain the second mapping relationship between the visual image coordinate system and the robot coordinate system.
[0090] The motion information of each joint of the first robotic arm 2 includes: the position and speed information of each joint.
[0091] Optionally, such as Figure 7 As shown, the navigation trolley calculates the spatial position information of the end effector of the first robotic arm 2 based on the joint motion information of each joint, the robot's kinematics, and the kinematic mapping relationship. During the puncture, the visual sensor tracks the target position at the end effector of the first robotic arm 2 in real time, adapting the end effector of the robotic arm to the robotic arm coordinate system and the visual sensor coordinate system, thereby achieving the fusion of intraoperative visual sensing and robot coordinate system.
[0092] Step 3: Based on the first mapping relationship and the second mapping relationship, the medical image and the visual image are fused to obtain the virtual operating space image.
[0093] The fusion of medical and visual images includes the fusion of the medical image coordinate system and the visual image coordinate system in step 1, and the fusion of the visual image coordinate system and the robot coordinate system in step 2. Based on steps 1 to 3, the fusion of pre-operation medical images, visual sensor images during puncture, and the robot coordinate system is achieved to obtain a virtual operating space image. In the virtual operating space image, the three-dimensional spatial position of the simulation instrument model 9 in the operating area can be calculated in real time.
[0094] In this embodiment, by fusing the medical images before puncture and the visual images during puncture, a virtual operating space image is obtained. During the puncture process, the simulation instrument model 9 is visualized in the virtual operating space image, showing the initial puncture position, puncture posture direction, and end target position of the target object. This solves the problem that in the prior art, the implantation status of the operating tool 3 in the current surgical field cannot be evaluated during the actual puncture of the target object.
[0095] In one embodiment, the puncture depth of the operating tool 3 cannot be visualized before the operating tool 3 punctures the target object. Therefore, to solve the above problem, in this embodiment, after obtaining the optimal target puncture position 6 and target puncture posture 7 relative to the target object, the control device controls the second robotic arm 8 to perform a pre-operation according to the target object, so that the simulation model carries the simulation instrument model to move and obtain the target puncture depth corresponding to the simulation instrument model.
[0096] The target puncture location 6 includes the initial puncture location and the final puncture location. For example, as... Figure 9As shown, point A represents the initial puncture position of the simulation instrument model 9, point B represents the final puncture position of the simulation instrument model 9, which is also the target lesion location, and vector AB represents the target puncture posture 7 of the simulation instrument model 9. The operator operates the second robotic arm 8, and through the motion boundary control algorithm of the second robotic arm 8, the second robotic arm 8 is restricted to moving only along the puncture posture of the simulation instrument model 9, and movement in other directions is blocked. Then, according to the master-slave mapping relationship between the simulation model and the second robotic arm 8, the simulation model is mapped to move along the axis direction, carrying the simulation instrument model 9 from point A to the target point B. During the movement of the simulation instrument model 9, if the simulation instrument model 9 avoids sensitive tissues and blood vessels and nerves, the current visualized puncture depth of the simulation instrument model 9 is taken as the target puncture depth of the simulation instrument model.
[0097] Optionally, after the control device obtains the optimal target puncture position 6 and target puncture posture 7 relative to the target object, it controls the end of the simulation instrument model 9 to move to the initial puncture position and target puncture posture 7 and keep them unchanged. The control device controls the second robotic arm 8 to perform a pre-operation, so that the simulation model carries the simulation instrument model 9 to move along the puncture posture direction within the motion boundary by the puncture depth. This allows the puncture depth of the simulation instrument model 9 along the puncture posture direction to be visually followed and adjusted based on the master-slave mapping relationship, until the end position of the simulation instrument model 9 reaches the puncture endpoint position along the puncture posture. The current puncture depth of the simulation instrument model is then taken as the target puncture depth corresponding to the simulation instrument model.
[0098] In some embodiments, the control device is also used to constrain the movement direction of the end of the second robotic arm 8 to be the same as the axial direction of the simulation device model.
[0099] Optionally, such as Figure 10 As shown, the control device keeps the initial puncture position and puncture posture of the simulation instrument model 9 unchanged. Based on the puncture posture of the simulation instrument model 9, the second robotic arm 8's posture is matched, and the operable direction axis of the second robotic arm 8 is calculated to obtain the operable target axis of the second robotic arm 8. The second robotic arm 8 is then controlled using an axial operating impedance control strategy and boundary force feedback direction. The second robotic arm 8 is restricted to free movement along the target axis. When the movement position of the second robotic arm 8 deviates from the target axis, the second robotic arm 8 receives boundary force feedback, and the force feedback direction is opposite to the deviation direction and proportional to the degree of deviation. Therefore, the operator is guided to operate the second robotic arm 8 along the target axis, and then the simulation instrument model 9 is moved along the axial direction by the master and slave operators to evaluate and determine the target puncture depth.
[0100] In this embodiment, before the control device controls the operating tool 3 to puncture the target object, the control device controls the second robotic arm 8 to perform a pre-operation according to the target object, locking the initial puncture position and puncture posture of the simulation instrument model 9, so that the simulation model carries the simulation instrument model 9 to adjust the puncture depth along the puncture posture direction, visually determine the target puncture depth, and avoid the problem of inaccurate positioning of the real operating tool 3.
[0101] In one embodiment, during the pre-operation controlled by the control device based on the target object, the control device constrains at least one parameter among the puncture position, puncture posture, and puncture depth, and controls the second robotic arm 8 to perform the pre-operation, causing the simulated instrument model to move under the remaining parameters of the puncture position, puncture posture, and puncture depth. Further, the puncture position includes an initial puncture position and an end-point puncture position. The control device is used to constrain the initial puncture position, move the simulated instrument model to obtain multiple end-point puncture positions, and plan multiple first puncture paths based on the initial puncture position and the multiple end-point puncture positions. The multiple first puncture paths are displayed on the target object image.
[0102] For example, when performing a biopsy for spinal bone cancer, it is often necessary to perform puncture biopsies on multiple lesion points in the lesion area. However, currently only the optimal needle insertion position for one puncture biopsy point can be determined at a time. To reduce the need for adjusting the position of the robotic arm during puncture, this embodiment provides a control mode for the simulation instrument model 9. According to this control mode, the initial puncture position of the robotic arm can satisfy the need for puncture biopsies on multiple lesion points in the lesion area. When switching the position of the lesion point during puncture, there is no need to adjust the position of the end of the robotic arm.
[0103] The control device controls the second robotic arm 8 to perform pre-operations based on the target object image (spine bone cancer lesion image), so that the simulation model carries the simulation instrument model to the initial puncture position. According to each puncture endpoint position in the puncture endpoint position group, the corresponding virtual puncture operation is performed, and the corresponding first puncture path is determined. The number of feasible paths that do not pass through the target object's non-target tissue in the first puncture path corresponding to the initial puncture position is counted. The initial puncture position is switched multiple times, and the number of feasible paths corresponding to the initial puncture position after each switch is obtained according to the above process. The target initial puncture position corresponding to the maximum number of feasible paths is determined. The target initial puncture position and the puncture posture corresponding to the maximum number of feasible paths are respectively used as the optimal initial puncture position and puncture posture orientation.
[0104] For example, such as Figure 11As shown, point A represents the initial puncture position of the simulation model, and point B represents the target lesion position, which is also the terminal target position of the simulation device model 9. The dashed lines from the initial puncture position A to different target lesion positions represent multiple first puncture paths, from which one of the first puncture paths is selected as the current first puncture path. If the simulation device model 9 does not pass through the normal tissue of the target object during implantation, a new first puncture path is selected as the current first puncture path, and the process returns to the step of keeping the initial puncture position of the simulation device model 9 unchanged, and continues until the simulation device model 9 implants the target object along all the first puncture paths; the number of feasible paths that do not pass through the non-target tissue of the target object in the first puncture path corresponding to the initial puncture position is counted, and the target puncture initial position corresponding to the maximum number of feasible paths and the puncture posture corresponding to the maximum number of feasible paths are respectively taken as the optimal puncture initial position and puncture posture.
[0105] Optionally, in a biopsy procedure, it is necessary to assess the initial puncture site to determine if it can cover the entire target lesion. In a biopsy procedure, such as... Figure 12 As shown, the second robotic arm 8 first moves freely, moving the simulation instrument model 9 to a selectable initial puncture position, and then keeps the initial puncture position of the simulation instrument model 9 unchanged. In this control mode, while the second robotic arm 8 operates, its posture joints can be adjusted, but its position joints cannot be actively operated. By adjusting the posture joints of the second robotic arm 8, the simulation instrument model 9 is controlled to perform puncture evaluation along different posture directions and paths. The evaluation assesses whether multiple virtual puncture paths from the current initial puncture position to multiple terminal target positions can cover all target lesion locations without touching normal tissue. If the requirements are met, the current initial puncture position is determined to be the optimal initial puncture position; if not, it moves to the next initial puncture position for evaluation until the optimal initial puncture position is found.
[0106] In this embodiment, in the scenario of puncture biopsy, the initial puncture position of the control simulation instrument model 9 remains unchanged, the puncture posture direction of the second robotic arm 8 within the motion boundary and the puncture depth along the puncture posture direction are adjusted, and the operation space under the current initial puncture position can be visually evaluated to see if it can cover multiple target lesion locations without damaging normal tissues and sensitive nerves. The optimal initial puncture position can also be visually and dynamically adjusted and determined to reduce the need for adjustment of the robotic arm position during puncture and to reduce damage to the target object.
[0107] In other embodiments of the present invention, the control device can also be used to constrain the puncture endpoint position, move the simulation instrument model to obtain multiple puncture initial positions, plan multiple second puncture paths based on the puncture endpoint position and multiple puncture initial positions, and display the multiple second puncture paths on the target object image.
[0108] During the puncture process using tool 3, after determining the location of the target lesion, it is necessary to find the optimal initial puncture position and target puncture posture 7 among multiple virtual puncture paths that can reach the unknown target lesion. Specifically, the control device determines the shortest target puncture path among the second puncture paths that do not pass through the target object's non-target tissue, and uses the initial puncture position and the final puncture position corresponding to the target puncture path as the optimal target puncture position 6, and uses the puncture posture corresponding to the target puncture path as the optimal target puncture posture 7.
[0109] Specifically, such as Figure 13 As shown, point B represents the location of the target lesion. Figure 13 The dashed lines from multiple selectable initial puncture positions to the same target lesion point represent multiple second puncture paths. One of these second puncture paths is selected as the current second puncture path. The control device uses the second robotic arm 8 to position the end of the simulation instrument model 9 at the puncture endpoint and maintain this position. The end position of the second robotic arm 8 must also remain unchanged. The posture of the second robotic arm 8 can change; that is, the Cartesian position velocity V0(V0_x,V0_y,V0_z) of the end of the second robotic arm 8 is 0. The corresponding mathematical formula is:
[0110]
[0111] In the formula, Js represents the Jacobian matrix corresponding to the second robotic arm 8, which is calculated based on the speed of the second robotic arm 8; This indicates the speed input of the corresponding joint of the second robotic arm 8.
[0112] The control strategy of the control device to keep the end position of the second robotic arm 8 unchanged is as follows: Figure 14As shown, the motion boundary control algorithm of the second robotic arm 8 keeps the end-effector position of the second robotic arm 8 constant, while its orientation can be changed, thereby adjusting the initial puncture position and puncture orientation of the simulation instrument model 9. Specifically, during operation of the second robotic arm 8, the posture joints of the second robotic arm 8 can be adjusted, while the position joints cannot be actively operated; they are used for follow-up compensation. If the velocity input of one posture joint affects the Cartesian position of the end-effector of the second robotic arm 8, the compensation joint velocity is calculated using the above formula to ensure that the Cartesian velocity of the end-effector of the second robotic arm 8 is 0. However, after obtaining the velocities of each joint of the second robotic arm 8, the joint velocity integral yields the joint command position. In this way, the end-effector position of the second robotic arm 8 remains constant during operation, thereby controlling the end-effector of the simulation instrument model 9 to adjust its orientation and needle insertion point around the end-effector.
[0113] If the simulation instrument model 9 does not pass through the normal tissue of the target object during implantation, the current virtual puncture path is determined to be a valid puncture path. A new virtual puncture path is selected as the current virtual puncture path, and the process returns to the step of keeping the end target position of the simulation instrument model 9 unchanged. The process continues until the simulation instrument model 9 implants the target object along all the second puncture paths. The shortest path among the second puncture paths that do not pass through the non-target tissue of the target object is the target puncture path. The initial puncture position and the end puncture position corresponding to the target puncture path are taken as the optimal target puncture position 6, and the puncture posture corresponding to the target puncture path is taken as the optimal target puncture posture 7.
[0114] If the simulation instrument model 9 passes through the normal tissue of the target object during the implantation of the target object along the current virtual puncture path, the current virtual puncture path is determined to be infeasible and is removed.
[0115] In this embodiment, the control device controls the puncture endpoint position of the simulation instrument model 9 to remain unchanged through the second robotic arm 8. The second robotic arm 8 controls the simulation instrument model 9 to visually and dynamically adjust the initial puncture position and puncture posture of the simulation instrument model 9, so that the simulation instrument model 9 is visually implanted into the target object along multiple second puncture paths. The virtual needle path that avoids sensitive tissues and blood vessels and nerves and has the shortest puncture path is taken as the optimal puncture path, which can improve the efficiency of the operation tool 3 puncture.
[0116] In some embodiments, such as Figure 15As shown, during the positioning or puncture process of the simulation instrument model 9 along the first or second puncture path, if the simulation instrument model 9 touches sensitive tissue or nerves and blood vessels, which may cause dangerous actions, the navigation image interface will issue a safety warning and simultaneously enhance the display on the navigation image interface to facilitate the operator's assessment of the safety and effectiveness of the operation path.
[0117] In some embodiments, the display device includes a projection unit, which is communicatively connected to a control device. The projection unit acquires and projects at least a simulation instrument model and an image of the target object onto the target object itself. The projection unit can be a projector or a wearable device, used to display a virtual model or image onto the target object, facilitating a comprehensive display of the puncture process in a combined virtual and real manner.
[0118] Based on the same inventive concept, embodiments of this application also provide a master-slave operation planning method, which can be applied to, for example... Figure 1 In the control device shown, such as Figure 16 As shown, the method includes the following steps:
[0119] Step 1602: Based on the target object, control the second robotic arm 8 to perform a pre-operation, so that the end of the simulation model of the second robotic arm 8 carries the simulation instrument model to move, so as to obtain the optimal target puncture position 6 and target puncture posture 7 relative to the target object; the simulation model and the second robotic arm 8 have a master-slave mapping relationship.
[0120] Pre-operation refers to the adjustments required by the second robotic arm 8 when controlling the movement of the simulated instrument model according to the master-slave mapping relationship. For example, controlling the second robotic arm 8 to translate or rotate along a horizontal line. The simulation model is the simulation structure of the second robotic arm 8 in the simulation environment, and the simulated instrument model is the simulation structure of the operating tool 3 in the simulation environment. The simulated instrument model is loaded at the end of the simulation model.
[0121] In this embodiment, a master-slave control scheme is adopted to control the movement of the simulation instrument model. Specifically, the second robotic arm 8 is the master operation end, and the simulation model is the slave operation end. The operator performs pre-operations through the second robotic arm 8 to make the simulation model carry the simulation instrument model to move, thereby controlling the puncture position and puncture posture of the simulation instrument model, and can adjust the puncture position and puncture posture of the simulation instrument model in real time and visually.
[0122] In some embodiments, the second robotic arm 8 includes three posture joints and three position joints. The axes of the three posture joints intersect at the end of the operating handle of the second robotic arm 8, that is, the movement of the posture joints of the second robotic arm 8 does not affect the end position of the second robotic arm 8.
[0123] Optionally, the control device controls the second robotic arm 8 to perform pre-operations based on the target object. Depending on the application scenario requirements, it can control the movement of the other two or one motion dimensions separately, based on one or two of the three motion dimensions of locking the puncture position, puncture posture and puncture depth. Once the corresponding motion dimension is locked, during the master-slave operation of the simulation instrument model 9, the motion in that direction will be restricted to remain stationary, while other motion dimensions will adapt to the motion. The master-slave control strategy and the master-end motion control will also adapt accordingly, and the end pose of the simulation model will be remapped according to the master-slave control strategy. Based on the master-slave mapping algorithm and the end pose of the simulation model, the control device, through the pre-operation of the second robotic arm 8, causes the end of the simulation model of the second robotic arm 8 to move carrying the simulation instrument model. If the simulation instrument model 9 passes through the normal tissue of the target object, it is determined that the current puncture position and current puncture posture of the simulation instrument model 9 are infeasible; if the simulation instrument model 9 does not pass through the normal tissue of the target object, it is determined that the current puncture position and current puncture posture of the simulation instrument model 9 are feasible, and the current puncture position and current puncture posture of the simulation instrument model 9 are taken as the target puncture position 6 and the target puncture posture 7.
[0124] Step 1604: Based on the target puncture position 6 and the target puncture posture 7, control the first robotic arm 2 to drive the operating tool 3 to position itself at the target puncture position 6 and the target puncture posture 7.
[0125] Optionally, the process by which the control device controls the first robotic arm 2 to move the operating tool 3 to the target puncture position 6 and the target puncture posture 7 is the same as the function of the control device in the above embodiment, and therefore will not be repeated here.
[0126] In this embodiment, the second robotic arm 8 includes a simulation model. The simulation model and the second robotic arm 8 have a master-slave mapping relationship. The end of the simulation model is used to load a simulation instrument model and control the second robotic arm 8 to perform pre-operations, causing the simulation instrument model at the end of the simulation model to move to obtain the optimal target puncture position 6 and target puncture posture 7 relative to the target object. In the above method, the operator can visually and intuitively evaluate the feasibility of the puncture position and puncture posture based on the current real operation scenario before the operation. Moreover, the visual master-slave teleoperation dynamically adjusts the puncture posture, puncture position, and puncture depth to determine the optimal target puncture position 6 and target puncture posture 7, and then guides the first robotic arm 2 to move to the optimal target puncture position 6 and target puncture posture 7, making the doctor's operation guidance more intuitive.
[0127] In one embodiment, controlling the second robotic arm 8 to perform a pre-operation based on the target object, causing the end effector of the simulation model of the second robotic arm 8 to move carrying the simulation device model, includes:
[0128] Constrain at least one of the parameters of puncture position, puncture posture, and puncture depth, and control the second robotic arm 8 to perform a pre-operation so that the simulated instrument model moves under the remaining parameters of puncture position, puncture posture, and puncture depth.
[0129] The puncture location includes the initial puncture position and the final puncture position. The second robotic arm 8 is controlled to perform pre-operations, including:
[0130] The initial puncture position is constrained, and the moving simulation instrument model obtains multiple puncture endpoint positions. Multiple first puncture paths are planned based on the initial puncture position and multiple puncture endpoint positions. The endpoint positions are constrained, and the moving simulation instrument model obtains multiple puncture initial positions. Multiple second puncture paths are planned based on the endpoint positions and multiple puncture initial positions.
[0131] The process of the control device controlling the second robotic arm 8 to perform pre-operations is the same as the function of the control device in the above system embodiment, so it will not be described again here.
[0132] In one embodiment, the method further includes the following steps:
[0133] Step 1: Obtain the image of the target object.
[0134] Step 2: Overlay the simulation device model onto the target object image.
[0135] Step 3: Display the simulation device model and the target object image onto the target object itself.
[0136] The operation of the display device overlaying the simulation instrument model onto the target object image and displaying the simulation instrument model and the target object image onto the target object body is the same as the operation of the display device in the above system, and will not be repeated here.
[0137] Based on the same inventive concept, this application also provides a puncture device for implementing the master-slave operation planning method described above. The solution provided by this device is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more puncture device embodiments for master-slave operations provided below can be found in the limitations of the master-slave operation planning method described above, and will not be repeated here.
[0138] In one embodiment, such as Figure 17 As shown, a master-slave puncture device is provided, comprising: a master-slave operation module 100 and an execution module 200, wherein:
[0139] The master-slave operation module 100 is used to control the second robotic arm 8 to perform pre-operations based on the target object, so that the end effector of the simulation model of the second robotic arm 8 carries the simulation instrument model to move, in order to obtain the optimal target puncture position 6 and target puncture posture 7 relative to the target object; the simulation model and the second robotic arm 8 have a master-slave mapping relationship; and
[0140] The execution module 200 is used to control the first robotic arm 2 to drive the operating tool 3 to position itself at the target puncture position 6 and the target puncture posture 7 based on the target puncture position 6 and the target puncture posture 7.
[0141] In one embodiment, the master-slave operation module 100 is further configured to: constrain at least one parameter among the puncture position, puncture posture, and puncture depth, and control the second robotic arm 8 to perform a pre-operation, so that the simulated instrument model moves under the remaining parameters of the puncture position, puncture posture, and puncture depth.
[0142] In one embodiment, the puncture location includes an initial puncture location and an end puncture location. The master-slave operation module 100 is further configured to: constrain the initial puncture location, move the simulation instrument model to obtain multiple end puncture locations, and plan multiple first puncture paths based on the initial puncture location and the multiple end puncture locations; constrain the end puncture location, move the simulation instrument model to obtain multiple initial puncture locations, and plan multiple second puncture paths based on the end puncture location and the multiple initial puncture locations.
[0143] In one embodiment, the puncture device further includes a display module 300 for acquiring an image of the target object;
[0144] Overlaying a simulated instrument model onto an image of the target object; and
[0145] Display the simulation device model and the target object image onto the target object itself.
[0146] The modules in the aforementioned master-slave puncture device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0147] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above embodiments.
[0148] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above embodiments.
[0149] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above embodiments.
[0150] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0151] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A master-slave puncture system, characterized in that, The master-slave puncture system includes: A first robotic arm and an operating tool assembled at the end of the first robotic arm; The second robotic arm is used to perform pre-operations. The second robotic arm includes a simulation model. The simulation model and the second robotic arm have a master-slave mapping relationship. The end of the simulation model is used to load the simulation device model. A display device is used to display an image of a target object and to overlay the simulation device model onto the image of the target object; and The control device is communicatively connected to the first robotic arm, the second robotic arm, and the display device respectively. The control device controls the second robotic arm to perform the pre-operation according to the target object, so that the simulation instrument model at the end of the simulation model moves to obtain the optimal target puncture position and target puncture posture relative to the target object. The control device also controls the first robotic arm to drive the operating tool to position itself at the target puncture position and target puncture posture according to the target puncture position and target puncture posture. The master-slave mapping process involves calculating the desired pose of the simulation device model's end effector based on the master-end Cartesian relative displacement, attitude input, and the initial position of the simulation model using a master-slave mapping control algorithm. Then, through the inverse kinematics of the robotic arm, the joint command positions of the simulation model are solved to obtain the command positions of the virtual joints. Finally, the simulation model is controlled to move the simulation device model by controlling the virtual joints.
2. The master-slave puncture system according to claim 1, characterized in that, The control device controls the second robotic arm to perform the pre-operation according to the target object, which also includes moving the simulation model carrying the simulation instrument model to obtain the target puncture depth corresponding to the simulation instrument model.
3. The master-slave puncture system according to claim 2, characterized in that, Obtaining the target puncture depth corresponding to the simulation instrument model further includes: constraining the movement direction of the end of the second robotic arm to be the same as the axial direction of the simulation instrument model.
4. The master-slave puncture system according to claim 1, characterized in that, It also includes a vision sensor, which is used to receive the real-time pose of the operating tool; The control device includes a first control unit and a second control unit that are communicatively connected. The first control unit is communicatively connected to the first robotic arm and the vision sensor; the second control unit is communicatively connected to the second robotic arm and the display device. The second control unit obtains and sends the target puncture position and the target puncture posture to the first control unit. The first control unit guides the first robotic arm to position the operating tool to the target puncture position and the target puncture posture based on the target puncture posture, the target puncture position, and the real-time pose.
5. The master-slave puncture system according to claim 4, characterized in that, The joint configuration of the first robotic arm is the same as that of the second robotic arm; the second control unit is also used to acquire the joint control parameters of the second robotic arm under the target puncture position and the target puncture posture; The first robotic arm and the second robotic arm are aligned by a calibration relationship. Based on the calibration relationship and the joint control parameters, the joints of the first robotic arm are controlled so that the operating tool at the end of the first robotic arm is positioned at the target puncture position and the target puncture posture.
6. The master-slave puncture system according to claim 1, characterized in that, The control device controls the second robotic arm to perform the pre-operation according to the target object, and the posture of the simulation device model loaded at the end of the simulation model remains fixed relative to the end of the simulation model.
7. The master-slave puncture system according to claim 6, characterized in that, The control device controls the second robotic arm to perform the pre-operation according to the target object. The control device constrains at least one parameter among the puncture position, puncture posture and puncture depth, and controls the second robotic arm to perform the pre-operation so that the simulation instrument model moves under the remaining parameters of puncture position, puncture posture and puncture depth.
8. The master-slave puncture system according to claim 7, characterized in that, The puncture location includes an initial puncture location and an end puncture location. The control device is used to constrain the initial puncture location, move the simulation instrument model to obtain multiple end puncture locations, and plan multiple first puncture paths based on the initial puncture location and the multiple end puncture locations. The multiple first puncture paths are displayed on the target object image. Alternatively, the control device is used to constrain the end puncture location, move the simulation instrument model to obtain multiple initial puncture locations, and plan multiple second puncture paths based on the end puncture location and the multiple initial puncture locations. The multiple second puncture paths are displayed on the target object image.
9. The master-slave puncture system according to claim 1, characterized in that, The display device includes a projection unit, which is communicatively connected to the control device. The projection unit acquires and projects at least the simulation device model and the target object image onto the target object body.
10. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: Based on the target object, the second robotic arm performs a pre-operation, causing the end effector of the second robotic arm's simulation model to move, and the simulation model is superimposed on the target object image to obtain the optimal target puncture position and target puncture posture relative to the target object. The simulation model and the second robotic arm have a master-slave mapping relationship; as well as Based on the target puncture position and the target puncture posture, the first robotic arm is controlled to move the operating tool to the target puncture position and the target puncture posture; The master-slave mapping process involves calculating the desired pose of the simulation device model's end effector based on the master-end Cartesian relative displacement, attitude input, and the initial position of the simulation model using a master-slave mapping control algorithm. Then, through the inverse kinematics of the robotic arm, the joint command positions of the simulation model are solved to obtain the command positions of the virtual joints. Finally, the simulation model is controlled to move the simulation device model by controlling the virtual joints.
11. The computer device according to claim 10, characterized in that, The step of controlling the second robotic arm to perform a pre-operation based on the target object, causing the end effector of the second robotic arm carrying the simulation device model to move, includes: By constraining at least one parameter among the puncture position, puncture posture, and puncture depth, the second robotic arm is controlled to perform the pre-operation, causing the simulated instrument model to move under the remaining parameters of puncture position, puncture posture, and puncture depth.
12. The computer device according to claim 11, characterized in that, The puncture location includes an initial puncture location and an end puncture location. Controlling the second robotic arm to perform the pre-operation includes: Constrain the initial puncture position, move the simulation instrument model to obtain multiple puncture endpoint positions, and plan multiple first puncture paths based on the initial puncture position and the multiple puncture endpoint positions; constrain the puncture endpoint positions, move the simulation instrument model to obtain multiple puncture initial positions, and plan multiple second puncture paths based on the puncture endpoint positions and the multiple puncture initial positions.
13. The computer device according to claim 10, characterized in that, When the processor executes the computer program, it also performs the following steps: Obtain the image of the target object; The simulated instrument model is overlaid onto the image of the target object; and Display the simulation device model and the target object image onto the target object itself.