Puncture surgical robot, control method thereof, controller, and storage medium

Abstract

The application discloses a puncture surgical robot and a control method, a controller and a storage medium thereof. The control method comprises the following steps: acquiring a first axial force borne by an ultrasonic probe through a force sensing module; calculating a second axial force borne by the ultrasonic probe along a central axis according to the first axial force and a preset force perception model; calibrating a normal vector of a contact point between the ultrasonic probe and the skin through an optical positioning acquisition module; calculating and processing the second axial force and the normal vector to obtain a normal pressure of the ultrasonic probe on the skin; controlling a mechanical arm to stop pressing down in the case that the normal pressure is greater than or equal to a force criterion threshold; and displaying an image of a puncture area to be collected by the ultrasonic image perception module and the optical positioning acquisition module in real time through a display module. According to the scheme of the embodiment of the application, the operation efficiency, operation accuracy and safety of the semi-automatic puncture surgery using the puncture surgical robot can be improved.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a puncture surgery robot and its control method, controller, and storage medium. Background Technology

[0002] With the advancement of robotics and the innovation of medical technology, the integration of medicine and robotics has become a development trend in the medical industry. Medical robots refer to various robotic devices used in surgery, medical training, rehabilitation, prostheses, and assistive devices for people with disabilities. Puncture surgery robots are a branch of medical surgical robots. Existing puncture surgery robots can be broadly categorized into fully automated and semi-automated puncture methods. Fully automated puncture involves directly fixing the needle to the end of a robotic arm, with the entire procedure controlled by the robot to insert the needle into the patient's body. This method has low physician involvement and a higher risk factor. Semi-automatic puncture primarily uses the robot for positioning during the procedure, with the final puncture requiring physician intervention. Compared to fully automated puncture, semi-automatic puncture offers greater physician involvement and lower risk. However, during semi-automatic puncture using a puncture surgery robot, it is difficult to control the contact pressure when the ultrasound probe is placed against the body. Insufficient pressure can lead to blurred ultrasound images, while excessive pressure can cause severe organ deformation. This makes it challenging to guarantee the accuracy and safety of puncture surgery using a puncture surgery robot. Summary of the Invention

[0003] The following is an overview of the topics described in detail in this article.

[0004] This invention provides a puncture surgery robot and its control method, controller, and storage medium, which can provide doctors with reliable operational reference information by utilizing the puncture surgery robot, thereby improving the operational efficiency, accuracy, and safety when using the puncture surgery robot for semi-automatic puncture surgery.

[0005] In a first aspect, embodiments of the present invention provide a puncture surgical robot, comprising:

[0006] robotic arm;

[0007] A puncture mechanism located at the end of the robotic arm;

[0008] The puncture sensing device is installed on the puncture mechanism, and the puncture sensing device includes a display module, an optical positioning and acquisition module, a force sensing module, and an ultrasonic image sensing module including an ultrasonic probe.

[0009] The controller is electrically connected to the robotic arm, the display module, the optical positioning and acquisition module, the ultrasonic image sensing module, and the force sensing module, respectively.

[0010] According to some embodiments of the present invention, the force sensing module includes at least two one-dimensional sensors.

[0011] Secondly, embodiments of the present invention provide a control method for a puncture surgical robot, applied to the puncture surgical robot as described in the first aspect, wherein the puncture surgical robot includes: a robotic arm; a puncture mechanism disposed at the end of the robotic arm; a puncture sensing device disposed on the puncture mechanism, the puncture sensing device including a display module, an optical positioning and acquisition module, a force sensing module, and an ultrasound image sensing module including an ultrasound probe; and a controller, the controller being electrically connected to the robotic arm, the display module, the optical positioning and acquisition module, the ultrasound image sensing module, and the force sensing module respectively;

[0012] Control methods include:

[0013] The first axial force on the ultrasonic probe is obtained through the force sensing module.

[0014] The second axial force on the ultrasonic probe along the central axis is calculated based on the first axial force and the preset force sensing model.

[0015] The normal vector of the contact point between the ultrasound probe and the skin is calibrated using the optical positioning and acquisition module.

[0016] The normal pressure of the ultrasound probe on the skin is obtained by calculation based on the second axial force and the normal vector.

[0017] If the normal pressure is greater than or equal to the force criterion threshold, control the robotic arm to stop pressing down;

[0018] The display module displays in real time images of the area to be punctured, acquired by the ultrasound image sensing module and the optical positioning acquisition module.

[0019] According to some embodiments of the present invention, the force sensing module includes at least two one-dimensional sensors; the acquisition of the first axial force on the ultrasonic probe through the force sensing module includes:

[0020] At least two axial force components are obtained through the at least two one-dimensional sensors;

[0021] The first axial force is obtained by adding the at least two axial component forces together.

[0022] According to some embodiments of the present invention, the step of calculating the second axial force on the ultrasonic probe along the central axis based on the first axial force and a preset force sensing model includes:

[0023] The preset force sensing model is set as follows: in, It is the first axial force. It is the second axial force, and k is a constant coefficient;

[0024] The measured first axial force is input into the preset force sensing model for calculation, and the second axial force is output.

[0025] According to some embodiments of the present invention, the step of calculating the normal pressure of the ultrasound probe on the skin based on the second axial force and the normal vector includes:

[0026] Determine the first angle between the second axial force and the normal vector;

[0027] The normal pressure of the ultrasound probe on the skin is calculated based on the cosine value of the second axial force and the first included angle.

[0028] According to some embodiments of the present invention, the image of the area to be punctured includes a lesion image and a location reference image, and the real-time display of the image of the area to be punctured acquired by the ultrasound image sensing module and the optical positioning acquisition module through the display module includes:

[0029] Ultrasound images of the skin are acquired through the ultrasound imaging sensing module.

[0030] The lesion image is obtained by performing image processing on the ultrasound image;

[0031] The optical positioning and acquisition module acquires the needle path at the puncture end of the puncture mechanism.

[0032] The display module displays the lesion image and the location reference image in real time. The location reference image is used to show the positional relationship between the needle path and the lesion on the skin.

[0033] According to some embodiments of the present invention, the control method further includes: controlling the robotic arm to continue pressing down when the normal pressure is less than a force criterion threshold.

[0034] Thirdly, embodiments of the present invention provide a controller, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the control method for the puncture surgical robot as described in the second aspect.

[0035] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the control method for the puncture surgical robot as described in the second aspect.

[0036] This invention includes the following embodiments: During a semi-automated puncture surgery, the puncture surgery robot first acquires the first axial force on the ultrasound probe through a force sensing module; then, it calculates the second axial force on the ultrasound probe along its central axis based on the first axial force and a preset force sensing model; subsequently, it calibrates the normal vector of the contact point between the ultrasound probe and the skin through an optical positioning acquisition module; next, it calculates the normal pressure of the ultrasound probe on the skin based on the second axial force and the normal vector; finally, when the normal pressure is greater than or equal to a force criterion threshold, it controls the robotic arm to stop pressing down, indicating to the doctor that the normal pressure generated by the current contact between the ultrasound probe and the patient's skin is appropriate, effectively reducing the probability of two situations: insufficient normal pressure leading to blurred ultrasound images, and excessive pressure leading to severe compression and deformation of the patient's organs; simultaneously, a display module displays in real time the images of the area to be punctured acquired by the ultrasound image sensing module and the optical positioning acquisition module, allowing the doctor to refer to, adjust, and operate the puncture mechanism to finally complete the puncture, thus realizing a semi-automated puncture surgery and improving the operational efficiency, accuracy, and safety of the puncture surgery. In other words, the embodiments of the present invention can provide doctors with reliable operational reference information by utilizing a puncture surgical robot, thereby improving the operational efficiency, operational accuracy and safety when using a puncture surgical robot to perform semi-automatic puncture surgery. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the system architecture of a puncture surgical robot 100 for performing a control method according to an embodiment of the present invention;

[0038] Figure 2 This is a front view of a puncture sensing device provided in one embodiment of the present invention;

[0039] Figure 3 This is a side view of a puncture sensing device provided in one embodiment of the present invention;

[0040] Figure 4 This is a side sectional view of a puncture sensing device provided in one embodiment of the present invention;

[0041] Figure 5 This is a schematic diagram of a force sensing module provided in one embodiment of the present invention.

[0042] Figure 6 This is a schematic diagram of a force sensing module provided in another embodiment of the present invention;

[0043] Figure 7 This is a flowchart of a control method for a puncture surgery robot provided in one embodiment of the present invention;

[0044] Figure 8 This is a schematic diagram of the discrete method used for the arc-shaped surface at the end of the ultrasonic probe provided in an embodiment of the present invention;

[0045] Figure 9 yes Figure 7 A flowchart illustrating the specific method of step S140;

[0046] Figure 10 This is a schematic diagram of the forces acting on an ultrasonic probe when it is in contact with human skin, according to one embodiment of the present invention.

[0047] Figure 11 This is a force diagram illustrating the movement of an ultrasonic probe along the insertion pin direction according to an embodiment of the present invention;

[0048] Figure 12 This is a force diagram illustrating the movement of an ultrasonic probe along its distal axis according to an embodiment of the present invention.

[0049] Figure 13 This is a schematic diagram of the forces acting on the ultrasonic probe when it is in contact with human skin, provided in another embodiment of the present invention;

[0050] Figure 14 This is a quantity-interval range statistical distribution chart provided in one embodiment of the present invention;

[0051] Figure 15 This is a schematic diagram of the hardware structure of a controller provided in one embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0053] It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0055] First, let's analyze some of the terms used in this application:

[0056] Optical positioning principle: For example, by capturing near-infrared light emitted from the same marker from different angles, and through relevant calculations and analysis, the three-dimensional spatial coordinates of each marker at different times can be obtained in real time and with precision.

[0057] This invention provides a puncture surgery robot and its control method, controller, and computer-readable storage medium. During a semi-automated puncture surgery, the robot first acquires the first axial force on the ultrasound probe via a force sensing module. Then, it calculates the second axial force on the ultrasound probe along its central axis based on the first axial force and a preset force sensing model. Next, it calibrates the normal vector of the contact point between the ultrasound probe and the skin using an optical positioning acquisition module. Then, it calculates the normal pressure of the ultrasound probe on the skin based on the second axial force and the normal vector. Finally, when the normal pressure is greater than or equal to a force criterion threshold, the robotic arm stops pressing down. Simultaneously, a display module displays in real-time images of the puncture area acquired by the ultrasound image sensing module and the optical positioning acquisition module. Therefore, this invention can provide doctors with reliable operational reference information by utilizing the puncture surgery robot, thereby improving the operational efficiency, accuracy, and safety of semi-automatic puncture surgery using the robot.

[0058] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0059] like Figure 1 As shown, the puncture surgical robot 100 includes: a robotic arm 110; a puncture mechanism 120 disposed at the end of the robotic arm 110; a puncture sensing device 130 disposed on the puncture mechanism 120, the puncture sensing device 130 including a display module 131, an optical positioning acquisition module 132, a force sensing module 133, and an ultrasound image sensing module 134 including an ultrasound probe 1341; and a controller 140, which is electrically connected to the robotic arm 110, the display module 131, the optical positioning acquisition module 132, the ultrasound image sensing module 134, and the force sensing module 133, respectively.

[0060] The puncture mechanism 120 is used to perform the puncture procedure.

[0061] The puncture sensing device 130 includes a display module 131, an optical positioning and acquisition module 132, a force sensing module 133, and an ultrasound image sensing module 134 including an ultrasound probe 1341. The display module 131 is used to display the image of the area to be punctured in real time, the optical positioning and acquisition module 132 is used to achieve optical positioning, the force sensing module 133 is used to output force feedback signals, and the ultrasound image sensing module 134 is used to acquire ultrasound images of the skin. By using the puncture sensing module, a mixed sensing of force, spatial position, and real-time image can be achieved in semi-automatic puncture surgery.

[0062] During a semi-automated puncture procedure, the controller 140 acquires the first axial force on the ultrasound probe 1341 via the force sensing module 133; then, it calculates the second axial force on the ultrasound probe 1341 along the central axis based on the first axial force and a preset force sensing model; subsequently, it calibrates the normal vector of the contact point between the ultrasound probe 1341 and the skin via the optical positioning acquisition module 132; next, it calculates the normal pressure of the ultrasound probe 1341 on the skin based on the second axial force and the normal vector; finally, it controls the robotic arm 110 to stop when the normal pressure is greater than or equal to a force criterion threshold. The pressure is stopped to indicate to the doctor that the normal pressure generated by the ultrasound probe 1341 in contact with the patient's skin is appropriate, effectively reducing the probability of two situations: insufficient normal pressure leading to blurred ultrasound images, and excessive pressure leading to severe compression and deformation of the patient's organs. At the same time, the display module displays in real time the images of the area to be punctured acquired by the ultrasound image sensing module 134 and the optical positioning acquisition module 132, so that the doctor can refer to them, adjust and operate the puncture mechanism 120 to finally complete the puncture action, realizing semi-automated puncture surgery and improving the operation efficiency, operation accuracy and safety of puncture surgery.

[0063] According to the embodiments provided by the present invention, the puncture surgical robot 100 can provide doctors with reliable operational reference information, thereby improving the operational efficiency, operational accuracy and safety when performing semi-automatic puncture surgery using the puncture surgical robot 100.

[0064] Based on the above system architecture, various embodiments of the puncture sensing device 130 of the present invention are proposed.

[0065] Combination Figure 1 , Figure 2 , Figure 3 and Figure 4 As shown, the puncture sensing device 130 includes a display module 131, an optical positioning and acquisition module 132, a force sensing module 133, and an ultrasonic image sensing module 134.

[0066] The ultrasound image sensing module 134 includes an ultrasound probe 1341 and an ultrasound probe holder 1342. The ultrasound probe 1341 and the ultrasound probe holder 1342 are connected, and the ultrasound probe holder 1342 is used to fix the ultrasound probe 1341 in place. The ultrasound probe 1341 is used to conform to the patient's skin to acquire ultrasound images of the skin, so that the controller 140 can perform image processing on the ultrasound images based on software algorithms to obtain lesion images.

[0067] The optical positioning acquisition module 132 includes a ball seat 1321 and an optical positioning ball 1322. The ball seat 1321 is used to hold the optical positioning ball 1322, and the ball seat 1321 and the optical positioning ball 1322 are connected. The optical positioning ball 1322 acquires the spatial coordinates of the end of the ultrasound probe 1341, so that the controller 140 can perform coordinate processing based on the optical positioning principle to obtain the movement path of the puncture end and the needle insertion path. Specifically, the movement path refers to the direction of movement of the ultrasound probe 1341 when it is moved, which can be along the end axis or along the needle insertion direction; the needle insertion path refers to the direction of the needle insertion hole of the puncture mechanism 120. Specifically, there are four optical positioning balls 1322. It is understood that the number of optical positioning balls 1322 can be set according to actual needs, and the present invention does not impose a specific limitation on this.

[0068] The display module includes a display screen 1311, which is communicatively connected to the controller 140. Under the control of the controller 140, it can display lesion images and location reference images acquired by the ultrasound image sensing module 134. The location reference image displays the positional relationship between the needle insertion path and the lesion on the skin. This allows the doctor to observe the positional relationship between the needle insertion path and the lesion in real time on the display screen 1311 during needle insertion, improving the efficiency, accuracy, and safety of the puncture surgery. It is understood that the display module includes one or more display screens 1311. When there are multiple display screens 1311, several can be set up independently of the puncture surgery robot 100, which is beneficial for multi-party consultation and surgical monitoring.

[0069] The force sensing module 133 includes two one-dimensional sensors 1331. The two one-dimensional sensors 1331 are respectively disposed on both sides of the ultrasonic probe 1341. Specifically, the force sensing module 133 also includes an end mounting bracket 1333 and fixing screws 1334. One end of the ultrasonic probe fixing clip 1342 is connected to the ultrasonic probe 1341, and the other end of the ultrasonic probe fixing clip 1342 is connected to the acquisition end of the two one-dimensional sensors 1331. The fixing end of the two one-dimensional sensors 1331 is connected to the end mounting bracket 1333 and fixed by the fixing screws 1334. When the ultrasound probe 1341 is in contact with the human body, the two one-dimensional sensors 1331 can dynamically measure the two axial components of the ultrasound probe 1341 along the axis on both sides. This allows the controller 140 to calculate the first axial force based on the two axial components. Then, based on the first axial force and the preset force sensing model, the controller calculates the second axial force on the ultrasound probe 1341 along the central axis. Finally, based on the second axial force, the controller calculates the normal pressure of the ultrasound probe 1341 against the skin. Thus, the controller judges in real time whether the degree of contact between the probe and the human body is reasonable by using the normal pressure and the force criterion threshold.

[0070] According to some embodiments of the present invention, the force sensing module 133 includes at least two one-dimensional sensors 1331. For example... Figure 1 As shown, the force sensing module 133 includes two one-dimensional sensors 1331.

[0071] In addition, such as Figure 5 As shown, the force sensing module 133 includes four one-dimensional sensors 1331, with two one-dimensional sensors 1331 respectively positioned on both sides of the ultrasound probe 1341. When the ultrasound probe 1341 is in contact with the human body, the four one-dimensional sensors 1331 can dynamically measure the four axial force components along the axial direction on both sides of the ultrasound probe 1341. This allows the controller 140 to calculate the first axial force based on the four axial force components, and further calculate the second axial force on the ultrasound probe 1341 along the central axis and the normal pressure of the ultrasound probe 1341 against the skin. It should be noted that the force sensing module 133 may include multiple one-dimensional sensors 1331 to obtain multiple axial force components when the ultrasound probe 1341 is in contact with the human body. It is understood that the number of one-dimensional sensors 1331 used in the force sensing module 133 can be set according to actual design requirements, and this invention does not impose specific limitations on this.

[0072] like Figure 6 As shown, the force sensing module 133 includes a six-dimensional sensor 1332, which replaces two one-dimensional sensors 1331. The six-dimensional sensor 1332 can also obtain the axial component force on both sides of the ultrasound probe 1341 along the axial direction when the ultrasound probe 1341 is in contact with the human body.

[0073] Those skilled in the art will understand that the system structure shown in the figures does not constitute a limitation on the embodiments of the present invention, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0074] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0075] It will be understood by those skilled in the art that the system architecture and application scenarios described in the embodiments of the present invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of the present invention, and do not constitute a limitation on the technical solutions provided in the embodiments of the present invention. It is known by those skilled in the art that with the evolution of system architecture and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present invention are also applicable to similar technical problems.

[0076] Based on the above system architecture and device structure, various embodiments of the control method of the puncture surgical robot of the present invention are proposed below.

[0077] like Figure 7 As shown, the control method of this puncture surgical robot can be applied to, for example... Figure 1 The system architecture of the puncture surgical robot shown may include, but is not limited to, steps S110 to S160.

[0078] Step S110: Obtain the first axial force on the ultrasonic probe through the force sensing module.

[0079] In this step, when it is not possible to directly obtain the force on the ultrasonic probe along the central axis, the first axial force on the ultrasonic probe along the central axis is indirectly obtained through the force sensing module. This is beneficial for providing a data basis for subsequent calculation of the second axial force and normal pressure on the ultrasonic probe along the central axis.

[0080] According to some embodiments of the present invention, the force sensing module includes at least two one-dimensional sensors; obtaining the first axial force on the ultrasonic probe through the force sensing module includes: obtaining at least two axial component forces through the at least two one-dimensional sensors; and adding the at least two axial component forces to obtain the first axial force.

[0081] In one embodiment, when the force sensing module includes two one-dimensional sensors, when the ultrasound probe is in contact with the human body, the two one-dimensional sensors can dynamically measure the two axial component forces along the axis on both sides of the ultrasound probe. and To make the controller based on two axial force components and Calculate the first axial force Specifically, in the case where the force sensing module includes two one-dimensional sensors, the first axial force for:

[0082]

[0083] In one embodiment, when the force sensing module includes four one-dimensional sensors, when the ultrasound probe is in contact with the human body, the four one-dimensional sensors can dynamically measure the four axial component forces along the axis on both sides of the ultrasound probe. and The controller calculates the first axial force based on the four axial components. Specifically, in the case where the force sensing module includes four one-dimensional sensors, the first axial force for:

[0084]

[0085] In one embodiment, when the force sensing module includes one six-dimensional sensor or multiple one-dimensional sensors, when the ultrasound probe is in contact with the human body, one six-dimensional sensor or multiple one-dimensional sensors can dynamically measure multiple axial force components of the ultrasound probe along the axial direction. Similarly, the first axial force can be obtained by calculation based on the multiple axial force components. The specific calculation method is the same as the one described above, and will not be repeated here.

[0086] Step S120: Calculate the second axial force on the ultrasonic probe along the central axis based on the first axial force and the preset force sensing model.

[0087] In this step, since the force exerted on the ultrasound probe along its central axis cannot be directly obtained, although the first axial force on the ultrasound probe along its central axis is indirectly obtained through a force sensing module, the actual axial force on the ultrasound probe along its central axis deviates from the first axial force due to factors such as probe material and size. Therefore, it is necessary to set a preset force sensing model, input the first axial force into the preset force sensing model, and output the second axial force. It can be understood that the second axial force is obtained after further optimization of the preset force sensing model, and is closer to the actual force exerted on the ultrasound probe along its central axis. This is beneficial for providing reliable reference data for subsequent judgments on whether to stop the robotic arm, thereby improving the accuracy and safety of the puncture operation.

[0088] According to some embodiments of the present invention, step S120 includes: setting a preset force sensing model as follows: in, It is the first axial force. It is the second axial force, and k is a constant coefficient; the measured first axial force is input into the preset force sensing model for calculation, and the second axial force is output.

[0089] Because the ultrasonic probe is made of a homogeneous material, the stress on its cross-section varies linearly, therefore... Where k is a constant coefficient, related to the probe material and size. To simplify the preset force sensing model, k can be assigned a value of 1. With k = 1, the preset force sensing model is: It is understandable that the constant coefficient k can be assigned a value based on the actual probe material and size. Therefore, this invention does not impose specific restrictions on the value of the constant coefficient k.

[0090] In one embodiment, when the force sensing module includes two one-dimensional sensors, it detects two axial component forces along the axial direction on both sides of the ultrasonic probe. and The first axial force was calculated. for: Then the second axial force at this time for:

[0091]

[0092] In one embodiment, when the force sensing module includes four one-dimensional sensors, the four axial component forces along the axial direction on both sides of the ultrasonic probe are measured. and The first axial force was calculated. for: Then the second axial force at this time for:

[0093]

[0094] In one embodiment, when the force sensing module includes one six-dimensional sensor or multiple one-dimensional sensors, the first axial force can be calculated based on the collected multiple axial component forces. The second axial force can then be calculated based on the preset force sensing model. The specific calculation method is the same as the above method, and will not be repeated here.

[0095] Step S130: The normal vector of the contact point between the ultrasound probe and the skin is calibrated using the optical positioning acquisition module.

[0096] In this step, the tip of the ultrasound probe is curved. When the ultrasound probe is in contact with the human body, this curved surface will be subjected to pressure from various directions. The normal vector of the contact point between the ultrasound probe and the skin is then calibrated. In order to obtain the second axial force With normal vector The first included angle θ between them facilitates the subsequent calculation of the normal pressure (i.e., the positive pressure) when the ultrasound probe is in contact with the human skin, which helps to improve the accuracy of the puncture operation.

[0097] In one embodiment, combined with Figure 8 To further explain step S130, specifically, in order to determine the normal vector of the contact point between the ultrasound probe and the human skin... The curved surface at the tip of the ultrasound probe is divided into a grid, with the grid intersections representing discrete points. Smaller grids result in denser discrete points. Based on this grid division method, the curved surface at the tip of the ultrasound probe is discretized into multiple dense discrete points. An optical positioning and acquisition module can then acquire the spatial coordinates of each point at the tip, thereby allowing the normal vector of each discrete point to be determined.

[0098] Step S140: Calculate the normal pressure of the ultrasound probe on the skin based on the second axial force and normal vector.

[0099] In this step, based on the calculated second axial force... and the normal vector obtained by calibration through the optical positioning acquisition module. The modulus of the normal pressure exerted by the ultrasound probe on the skin was calculated. Normal pressure reflects the degree of contact between the ultrasound probe and the skin.

[0100] Step S150: When the normal pressure is greater than or equal to the force criterion threshold, control the robotic arm to stop pressing down.

[0101] In this step, the normal pressure The force is compared with the force criterion threshold F to determine the normal pressure. When the force criterion threshold F is greater than or equal to the force threshold, the probe stops pressing down when the robotic arm stops pressing down, indicating that the normal pressure is appropriate. At the same time, the robotic arm stops pressing down to indicate to the doctor that the normal pressure is appropriate and the next manual operation can be performed. This effectively reduces the probability of two situations: the normal pressure is too small, resulting in blurred ultrasound images, and the pressure is too large, resulting in severe compression and deformation of the patient's organs. This improves the efficiency, accuracy and safety of semi-automatic puncture surgery.

[0102] According to some embodiments of the present invention, the control method further includes: if the normal pressure is less than the force criterion threshold, controlling the robotic arm to continue pressing down, then the probe continues to press down; achieving partial automated control, which is beneficial to improving the operational efficiency of semi-automatic puncture surgery.

[0103] Understandably, the force criterion threshold F can be set to 255.

[0104] Specifically, the force criterion threshold F can be determined through multiple experimental tests. Under the premise of ensuring clear ultrasound images and human comfort, the force criterion threshold F is determined through experimental testing, and the force-sensing-based robotic arm operation judgment criteria are established. Then the control probe stops pressing down; otherwise, Then the control probe continues to press down.

[0105] Step S160: The display module displays the images of the area to be punctured acquired by the ultrasound imaging sensing module and the optical positioning acquisition module in real time.

[0106] In this step, after the ultrasound imaging sensing module and the optical positioning acquisition module acquire images of the area to be punctured, the images of the area to be punctured are displayed in real time on the display module, which is convenient for doctors to refer to, adjust and operate the puncture mechanism to finally complete the puncture action, thereby improving the accuracy and safety of the puncture surgery.

[0107] Specifically, the image of the area to be punctured includes the lesion image and the location reference image.

[0108] According to some embodiments of the present invention, step S160 includes: acquiring an ultrasound image of the skin through an ultrasound image sensing module; performing image processing on the ultrasound image to obtain a lesion image; acquiring the needle path of the puncture end of the puncture mechanism through an optical positioning acquisition module; and displaying the lesion image and a position reference image in real time through a display module, wherein the position reference image is used to display the positional relationship between the needle path and the lesion on the skin.

[0109] In this embodiment of the invention, by employing a control method for a puncture surgical robot including the steps S110 to S160 described above, during a semi-automated puncture surgery, the puncture surgical robot first acquires the first axial force on the ultrasound probe through a force sensing module; then, it calculates the second axial force on the ultrasound probe along its central axis based on the first axial force and a preset force sensing model; subsequently, it calibrates the normal vector of the contact point between the ultrasound probe and the skin through an optical positioning acquisition module; next, it calculates the normal pressure of the ultrasound probe on the skin based on the second axial force and the normal vector; finally, when the normal pressure is greater than or equal to... Under the force threshold condition, the robotic arm stops pressing down, indicating to the doctor that the normal pressure generated by the ultrasound probe's contact with the patient's skin is appropriate. This effectively reduces the probability of two situations: insufficient normal pressure leading to blurred ultrasound images, and excessive pressure causing severe compression and deformation of the patient's organs. Simultaneously, the display module shows real-time images of the area to be punctured, acquired by the ultrasound image sensing module and optical positioning acquisition module, allowing the doctor to refer to and adjust the puncture mechanism to complete the puncture, achieving semi-automatic puncture surgery and improving the efficiency, accuracy, and safety of the procedure. In other words, this invention provides reliable operational reference information to doctors using a puncture surgery robot, thereby improving the efficiency, accuracy, and safety of semi-automatic puncture surgery performed with a puncture surgery robot.

[0110] Reference Figure 9 According to some embodiments of the present invention, step S140 may include, but is not limited to, steps S210 to S220.

[0111] Step S210: Determine the first angle between the second axial force and the normal vector;

[0112] Step S220: Calculate the normal pressure of the ultrasound probe on the skin based on the cosine value of the second axial force and the first included angle.

[0113] The normal pressure is calculated through steps S210 to S220, providing a control basis for the operation of the automatically controlled robotic arm.

[0114] Specifically, after determining the first angle θ between the second axial force and the normal vector, the magnitude of the second axial force is calculated. The magnitude of the second axial force The normal pressure value is obtained by multiplying the normal pressure of the ultrasound probe on the skin by the cosine of the first included angle θ. In other words, the formula for calculating the normal pressure of the ultrasound probe on the skin is:

[0115] In one embodiment, such as Figure 10As shown, in the case where the force sensing module includes two one-dimensional sensors, assuming that at time t, the ultrasonic probe 1341 is just in contact with the human skin, the normal vector is calibrated at the contact point. Because the ultrasonic probe itself has a certain weight, the sensor will be subjected to a certain amount of tension. Therefore, here and It is directional. The force analysis of the ultrasonic probe and the two sensors is as follows: Figure 10 As shown, therefore, according to Newton's law, the normal pressure at the contact point can be obtained. Wherein, the first included angle θ is and The included angle, due to and Parallel, the first included angle θ is also known as and The included angle.

[0116] like Figure 11 and Figure 12 As shown, the downward velocity of the ultrasonic probe at time t-δt. The procedure can be provided by a puncture surgical robot system (which allows selection of whether the puncture tip moves along the direction of the needle insertion or along the tip axis as it approaches the human body), while the human skin surface model can be provided by a digital human body created preoperatively using methods such as Maker microsphere optical positioning. Therefore, the depressurization velocity of each discrete point along the probe can be calculated. The direction is to reach the point closest to the human skin. Assuming the discrete points are sufficiently dense, this point closest to the skin is the discrete point that will first contact the skin during the actual pressing process.

[0117] If the mechanical end effector moves along the direction of the insertion needle as it approaches the human body, then there is a second angle α between the direction of movement and the end effector axis. The shortest distance D between the ultrasound probe and the skin can then be calculated. min =d1 (e.g.) Figure 11 (As shown); If the mechanical end effector moves along the end effector axis as it approaches the human body, then the direction of movement is parallel to the end effector axis, and the shortest distance D between the ultrasound probe and the skin is... min =d2 (e.g.) Figure 12 As shown in the figure, the corresponding discrete point of the probe is the contact point at time t, and thus the normal of the discrete point of the probe is the normal of the contact point. Finally, by calculating the axial direction of the ultrasonic probe ( (direction) and contact point normal ( The angle between the directions is used to obtain the first angle θ.

[0118] In summary, the normal pressure at the contact point We can obtain:

[0119]

[0120] like Figure 13 As shown, when the force sensing module includes four one-dimensional sensors, the force analysis of the ultrasonic probe and the four sensors is as follows: Figure 13 As shown, therefore, according to Newton's law, the normal pressure at the contact point can be obtained:

[0121]

[0122] Wherein, the first included angle θ is and The included angle, also known as the first included angle θ, is and The angle between the two sensors. When there are multiple one-dimensional sensors, the method for calculating the normal pressure is as shown above, and will not be repeated here.

[0123] Here is an example illustrating the specific implementation method of the experimental test force criterion threshold F.

[0124] Using a body membrane to simulate the human body, six lesion points were set at different locations within the membrane. Three different needle insertion paths were planned for each lesion point. The puncture surgical robot was controlled to move towards the body membrane along the needle direction or the end-effector axis. Finally, the puncture surgical robot was fine-tuned to ensure the ultrasound probe at its end adhered to the human skin. Visual observation showed that if the ultrasound was clearly visible, the lesion inside the body membrane was present, and the ultrasound probe did not deform the body membrane, then the adhesion pressure was appropriate. When the force sensing module included two one-dimensional sensors, the values ​​from both sensors were recorded. and and the second axial force With normal vector The first included angle θ between them. Where, and This is the difference between the two sensors in the robotic arm's hovering state and after it has pressed down to adhere to the membrane. The robotic arm's hovering state refers to the state where the robotic arm stops above the membrane after moving along the planned path; the next step is to move further along the direction of the insertion pin or the end-effector axis to adhere to the membrane. The solution process for the first included angle θ has been clearly described above and can be directly read from the software calculation. The numerical value... and Substituting the first included angle θ into the formula: In the middle, the calculation is obtained After multiple tests, several data points were obtained. Multiple data Organized as follows Figure 14 The chart shows the quantity-interval value statistical distribution. From... Figure 14 It can be seen from this that The values ​​in the 255-265 range account for the largest proportion, therefore the force criterion threshold is set at 255. Then... The probe stopping its downward pressure was then added as a condition to the controller's program for controlling the robotic arm, and the experiment was repeated. This time, however, the robotic arm was automatically stopped by software detecting the force feedback signal, without manual fine-tuning. After stopping, the clarity of the ultrasound probe and the degree of skin compression were observed to further verify the correctness of the force criterion threshold. The final force criterion threshold was determined to be 255. It is understandable that, in the case of a force sensing module including four or more one-dimensional sensors or one six-dimensional sensor, multiple [force criterion thresholds] can be calculated in the same way described above. The output threshold is then determined, which will not be elaborated here.

[0125] Reference Figure 15 The controller 140 includes a memory 1520, a processor 1510, and a computer program stored in the memory 1520 and executable on the processor. When the processor 1510 executes the computer program, it implements the control method of the puncture surgery robot described above.

[0126] The processor 1510 and memory 1520 can be connected via a bus or other means.

[0127] The processor 1510 can be implemented using a general-purpose central processing unit, microprocessor, application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0128] Memory 1520, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, the memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 1520 may optionally include memory remotely located relative to the processor, and this remote memory can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0129] The non-transient software program and instructions required to implement the control method of the puncture surgical robot in the above embodiments are stored in memory. When executed by a processor, the control method of the puncture surgical robot in the above embodiments is executed, for example, the method described above is executed. Figure 7 and Figure 9 The method steps are shown.

[0130] An embodiment of the present invention also provides a computer-readable storage medium storing computer-executable instructions that are executed by a processor or controller, for example, by a processor in the above-described device embodiments, causing the processor to perform the control method of the puncture surgical robot in the above embodiments, for example, performing the above-described... Figure 7 and Figure 9 The method steps are shown.

[0131] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0132] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the present invention.

Claims

1. A puncture surgery robot, characterized in that, include: robotic arm; A puncture mechanism located at the end of the robotic arm; The puncture sensing device is installed on the puncture mechanism, and the puncture sensing device includes a display module, an optical positioning and acquisition module, a force sensing module, and an ultrasonic image sensing module including an ultrasonic probe. The controller is electrically connected to the robotic arm, the display module, the optical positioning and acquisition module, the ultrasonic image sensing module, and the force sensing module, respectively. The controller is configured to implement a control method for a puncture surgical robot. The control method includes: The first axial force on the ultrasonic probe is obtained through the force sensing module. The second axial force on the ultrasonic probe along the central axis is calculated based on the first axial force and the preset force sensing model. The spatial coordinates of discrete points on the end surface of the ultrasound probe are obtained through the optical positioning and acquisition module, and the normal vector of the contact point between the ultrasound probe and the skin is calibrated based on the spatial coordinates of the discrete points; wherein, the contact point is the point closest to the human skin along the direction of the downward pressure velocity of the ultrasound probe. The normal pressure of the ultrasound probe on the skin is obtained by calculation based on the second axial force and the normal vector. If the normal pressure is greater than or equal to the force criterion threshold, control the robotic arm to stop pressing down; The display module displays images of the area to be punctured in real time, acquired by the ultrasound image sensing module and the optical positioning acquisition module. The step of calculating the second axial force on the ultrasonic probe along its central axis based on the first axial force and the preset force sensing model includes: The preset force sensing model is set as follows: ,in, It is the first axial force. It is the second axial force, and k is a constant coefficient related to the material and size of the ultrasonic probe; The first axial force is input into the preset force sensing model for calculation, and the second axial force is output.

2. The puncture surgical robot according to claim 1, characterized in that, The force sensing module includes at least two one-dimensional sensors.

3. A controller, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a control method for a puncture surgical robot. The puncture surgical robot comprises: a robotic arm; a puncture mechanism disposed at the end of the robotic arm; a puncture sensing device disposed on the puncture mechanism, the puncture sensing device including a display module, an optical positioning and acquisition module, a force sensing module, and an ultrasound image sensing module including an ultrasound probe; and a controller electrically connected to the robotic arm, the display module, the optical positioning and acquisition module, the ultrasound image sensing module, and the force sensing module. The control method includes: The first axial force on the ultrasonic probe is obtained through the force sensing module. The second axial force on the ultrasonic probe along the central axis is calculated based on the first axial force and the preset force sensing model. The spatial coordinates of discrete points on the end surface of the ultrasound probe are obtained through the optical positioning and acquisition module, and the normal vector of the contact point between the ultrasound probe and the skin is calibrated based on the spatial coordinates of the discrete points; wherein, the contact point is the point closest to the human skin along the direction of the downward pressure velocity of the ultrasound probe. The normal pressure of the ultrasound probe on the skin is obtained by calculation based on the second axial force and the normal vector. If the normal pressure is greater than or equal to the force criterion threshold, control the robotic arm to stop pressing down; The display module displays images of the area to be punctured in real time, acquired by the ultrasound image sensing module and the optical positioning acquisition module. The step of calculating the second axial force on the ultrasonic probe along its central axis based on the first axial force and the preset force sensing model includes: The preset force sensing model is set as follows: ,in, It is the first axial force. It is the second axial force, and k is a constant coefficient related to the material and size of the ultrasonic probe; The first axial force is input into the preset force sensing model for calculation, and the second axial force is output.

4. The controller according to claim 3, characterized in that, The force sensing module includes at least two one-dimensional sensors; the acquisition of the first axial force on the ultrasonic probe through the force sensing module includes: At least two axial force components are obtained through the at least two one-dimensional sensors; The first axial force is obtained by adding the at least two axial component forces together.

5. The controller according to claim 3, characterized in that, The step of calculating the normal pressure of the ultrasound probe on the skin based on the second axial force and the normal vector includes: Determine the first angle between the second axial force and the normal vector; The normal pressure of the ultrasound probe on the skin is calculated based on the cosine value of the second axial force and the first included angle.

6. The controller according to claim 3, characterized in that, The image of the area to be punctured includes a lesion image and a location reference image. The real-time display of the image of the area to be punctured, acquired by the ultrasound image sensing module and the optical positioning acquisition module, via the display module includes: Ultrasound images of the skin are acquired through the ultrasound imaging sensing module. The lesion image is obtained by performing image processing on the ultrasound image; The optical positioning and acquisition module acquires the needle path at the puncture end of the puncture mechanism. The display module displays the lesion image and the location reference image in real time. The location reference image is used to show the positional relationship between the needle path and the lesion on the skin.

7. The controller according to claim 3, characterized in that, The control method further includes controlling the robotic arm to continue pressing down when the normal pressure is less than the force criterion threshold.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the control method of the puncture surgical robot according to any one of claims 3 to 7.

Images