Assisted puncture device, puncture system and computer readable storage medium

Abstract

The application provides an auxiliary puncture device, a puncture system and a computer readable storage medium. The auxiliary puncture device comprises a display module, a processor and a memory. The memory is used for storing a calculation program. The processor is used for calling the calculation program in the memory to execute the following steps: obtaining a plurality of tomographic images obtained by a medical imaging device in a scanning operation on a puncture object; obtaining puncture state information of a puncture needle in a surgical robot; determining position information of a needle tip point of the puncture needle and position information of a predicted target point according to the puncture state information; marking the needle tip point and the predicted target point on a first tomographic image according to the position information of the needle tip point and the position information of the predicted target point, to obtain a marked first tomographic image; and displaying the marked first tomographic image through the display module. By marking the needle tip point and the predicted target point on the tomographic image for display, the dependence on the experience of doctors can be reduced.

Description

Technical Field

[0001] This application belongs to the field of medical device technology, and in particular relates to an auxiliary puncture device, a puncture system and a computer-readable storage medium. Background Technology

[0002] Percutaneous needle insertion into lesions or soft tissue is a crucial component of modern clinical medicine and has attracted considerable attention from researchers in recent years. Many clinical applications, such as biopsy, injections, neurosurgery, and cancer treatment, utilize similar minimally invasive techniques.

[0003] Current puncture procedures mainly include manual blind puncture, real-time image-guided puncture, and master-slave real-time guided puncture. Master-slave real-time guided puncture refers to a procedure where the surgeon remotely operates a surgical robot outside the operating room (i.e., a CT-shielded room) to perform the puncture, and simultaneously controls the CT (Computed Tomography) equipment to scan and display images of the patient in real time. In this approach, because the surgeon operates remotely from outside the operating room, they can avoid radiation exposure during the procedure.

[0004] However, the master-slave real-time guided puncture protocol also has certain problems. During the puncture process, the physician needs to rely on clinical experience to judge the puncture path and the puncture result after posture adjustment, making it difficult to guarantee that the puncture needle reaches the optimal puncture point after posture adjustment. In other words, because the current master-slave real-time guided puncture protocol relies too much on the physician's subjective medical experience, it is prone to problems such as incomplete puncture, which in turn leads to repeated posture adjustments, scans, and punctures during the operation, prolonging the operation time. Multiple posture adjustments, scans, and punctures also increase the patient's pain and radiation dose.

[0005] Therefore, how to reduce the dependence of master-slave real-time guided puncture protocols on physician experience, thereby improving the puncture accuracy of master-slave real-time guided puncture protocols, has become an urgent technical problem to be solved. Summary of the Invention

[0006] This application provides an auxiliary puncture device, a puncture system, and a computer-readable storage medium, which can reduce the dependence of master-slave real-time guided puncture schemes on physician experience, thereby improving the puncture accuracy of master-slave real-time guided puncture schemes.

[0007] In a first aspect, embodiments of this application provide an auxiliary puncture device, comprising: a display module and a processor. The processor is configured to perform the following steps: acquiring multiple tomographic images obtained by a medical imaging device scanning a puncture object; acquiring puncture status information of a puncture needle in a surgical robot, the puncture status information including the status information of the puncture needle when the medical imaging device scans the puncture object; determining the position information of the needle tip and the position information of a predicted target point based on the puncture status information, the predicted target point being the spatial point reached by the needle tip after the puncture needle completes insertion in its current posture; marking the needle tip and the predicted target point on a first tomographic image based on the position information of the needle tip and the position information of the predicted target point, thereby obtaining a marked first tomographic image, the first tomographic image including at least one or two of the multiple tomographic images; and displaying the marked first tomographic image through the display module.

[0008] The auxiliary puncture device in the first aspect marks the needle tip and the predicted target point on the tomographic image and displays the marked tomographic image, so that the doctor can clearly know the position of the needle tip and the position of the predicted target point that the needle tip can reach. This can help the doctor to judge the needle tip and the puncture point that the puncture needle can reach, reduce the dependence on the doctor's experience, and improve the accuracy of puncture.

[0009] In one possible implementation of the first aspect, the processor further performs the following steps: annotating the target lesion of the puncture object on a second tomographic image among multiple tomographic images to obtain an annotated second tomographic image, the second tomographic image including tomographic images of the target lesion scanned from the multiple tomographic images; and displaying the annotated second tomographic image through a display module. In this implementation, by annotating the target lesion, the user can more clearly see the lesion of the puncture object, thereby enabling the user to have more precise control over the puncture process and improving the user experience.

[0010] In one possible implementation of the first aspect, the processor further performs the following steps: while displaying the annotated second fault image through the display module, it also displays adjacent fault images of the second fault image through the display module.

[0011] In one possible implementation of the first aspect, the processor further performs the following steps: displaying a three-dimensional image of the puncture needle and a three-dimensional image of the target lesion via a display module, wherein the three-dimensional images of the puncture needle and the target lesion are obtained separately using multiple tomographic images. In this implementation, displaying the three-dimensional images of the puncture needle and the target lesion makes the presentation of the puncture needle and the target lesion more three-dimensional and vivid.

[0012] In one possible implementation of the first aspect, the puncture needle and the target lesion are displayed in the same three-dimensional image. The relative positional relationship between the puncture needle and the target lesion in the same three-dimensional image is the same as their relative positional relationship in the image coordinate system, which is the coordinate system of multiple tomographic images. In this implementation, by displaying the puncture needle and the target lesion in the same three-dimensional image and accurately presenting their relative positional relationship, the user can more intuitively see the puncture progress, thus facilitating the user's judgment on whether the puncture has been completed.

[0013] In one possible implementation of the first aspect, the puncture status information includes the position information of the needle tip, which is calculated based on the relative positional relationship between the puncture needle and the surgical robot. The processor further performs the following steps: extracting the needle path contour of the puncture needle based on multiple tomographic images, the needle path contour representing the three-dimensional structure of the puncture needle; determining the actual position information of the needle tip of the puncture needle based on the needle path contour, the actual position information of the needle tip representing the position of the needle tip within the needle path contour; if an abnormality is determined in the needle path contour based on the position information of the needle tip and the actual position information of the needle tip, a needle path abnormality warning is generated. In this implementation, the needle path abnormality warning can promptly alert the user to abnormal information of the needle path, avoiding the impact of needle path abnormalities on puncture accuracy.

[0014] In one possible implementation of the first aspect, the scanning operation is performed during the orientation adjustment process or during the needle insertion process, the orientation adjustment process including adjusting the posture of the puncture needle, and the needle insertion process including adjusting the puncture depth of the puncture needle.

[0015] In one possible implementation of the first aspect, when the needle tip and the predicted target point are annotated on the same tomographic image, the first tomographic image after annotation also displays a line connecting the needle tip and the predicted target point. In this implementation, the line can indicate the direction of the puncture path when the puncture needle is puncturing in the current puncture state, allowing the user to more clearly determine the location of the puncture path and thus more accurately determine whether to perform puncture based on the current puncture state.

[0016] In one possible implementation of the first aspect, the processor further performs the following steps: while displaying the annotated first fault image through the display module, it also displays adjacent fault images of the first fault image through the display module.

[0017] In one possible implementation of the first aspect, the processor further performs the following steps: the display module also displays puncture status information, which includes at least one of puncture depth, puncture speed and posture angle, wherein the puncture depth represents the depth to which the tip of the puncture needle enters the skin, the posture angle represents the angle of the puncture needle relative to the puncture object, and the puncture speed represents the speed at which the puncture needle moves along the needle insertion direction.

[0018] In one possible implementation of the first aspect, the processor further performs the following steps: if a first orientation adjustment command is received, the processor identifies the first location information of the target lesion from multiple tomographic images, and the first orientation adjustment command instructs the auxiliary puncture device to automatically adjust its orientation; based on the first location information and puncture status information, the processor obtains first adjustment data, which represents the posture change data of the puncture needle when the predicted target point is adjusted to the first position corresponding to the first location information; and sends a first movement command corresponding to the first adjustment data to the surgical robot, which controls the surgical robot to move the puncture needle for orientation adjustment. In this implementation, the auxiliary puncture device obtains the location of the target lesion by performing image recognition on multiple tomographic images, where the location of the target lesion is the first location information, and obtains the first adjustment data based on the first location information and puncture status information. This method enables automatic orientation adjustment, saving user orientation adjustment time and improving orientation adjustment accuracy.

[0019] In one possible implementation of the first aspect, before sending the first movement command corresponding to the first adjustment data to the surgical robot, the processor performs the following steps: if the processor determines that the puncture depth of the puncture needle is greater than a first threshold, or the predicted distance between the target point and the first position is greater than a second threshold, or direct posture adjustment based on the first adjustment data would cause interference between the puncture needle and the target tissue or organ of the puncture object, then a second movement command is sent to the surgical robot. The second movement command is used to control the surgical robot to move the puncture needle to withdraw it first. In this implementation, controlling the surgical robot to withdraw the needle before adjusting its posture under the above three conditions can significantly reduce damage to the puncture object and improve the safety of automatic posture adjustment.

[0020] In one possible implementation of the first aspect, before sending the first movement instruction corresponding to the first adjustment data to the surgical robot, which is used to control the surgical robot to move the puncture needle for posture adjustment, the processor further performs the following steps: displaying the first adjustment data through a display module; and if an instruction to use the first adjustment data is received, sending the first movement instruction corresponding to the first adjustment data to the surgical robot. In this implementation, user confirmation is required before specific posture adjustment can be performed during automatic posture adjustment, which is equivalent to adding a doctor's review and confirmation step, thereby improving the reliability of the puncture method and the accuracy of the puncture results.

[0021] In one possible implementation of the first aspect, the processor further performs the following steps: if a second orientation adjustment instruction is received, a manual orientation adjustment instruction is sent to the surgical robot, wherein the manual orientation adjustment instruction is an instruction for the surgical robot to control the orientation of the puncture needle in response to a user operation; the second orientation adjustment instruction instructs the user to select manual orientation adjustment. In this implementation, the provided manual orientation adjustment process can meet the user's need for a high degree of autonomy, facilitating manual orientation adjustment by the user.

[0022] In one possible implementation of the first aspect, the scanning operation is automatically controlled by an auxiliary puncture device to perform the medical imaging equipment. When the auxiliary puncture device automatically controls the medical imaging equipment to perform the scanning, the processor executes the following steps: if a prompt message from the surgical robot indicating that a puncture operation is to be performed is received, the processor controls the medical imaging equipment to perform the scanning operation. The puncture operation includes an orientation adjustment operation or a needle insertion operation. The orientation adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle. In this implementation, by automatically controlling the medical imaging equipment to perform the scanning operation through an auxiliary puncture device, the timeliness of the scanning operation is improved, thereby ensuring the safety of the puncture.

[0023] Secondly, embodiments of this application provide a puncture system, comprising: a medical imaging device, a surgical robot, and an auxiliary puncture device as described in any one of the first aspects, wherein: the medical imaging device is used to scan the puncture object, obtain multiple tomographic images, and send the multiple tomographic images to the auxiliary puncture device; the surgical robot is used to send the puncture status information of the puncture needle in the surgical robot to the auxiliary puncture device, the puncture status information including the status information of the puncture needle when the medical imaging device scans the puncture object; the auxiliary puncture device is used to perform puncture... The status information determines the position information of the needle tip and the position information of the predicted target point. The predicted target point is the spatial point reached by the needle tip after the needle is inserted according to the posture in the puncture status information. The auxiliary puncture device is used to mark the needle tip and the predicted target point on the first tomographic image according to the position information of the needle tip and the position information of the predicted target point, so as to obtain the marked first tomographic image. The first tomographic image includes at least one or two tomographic images. The auxiliary puncture device is used to display the marked first tomographic image through the display module in the auxiliary puncture device.

[0024] In one possible implementation of the second aspect, the surgical robot includes: a main surgical robot hand, a first processor, and a secondary surgical robot hand. A puncture needle is disposed on the secondary surgical robot hand, and the main surgical robot hand is communicatively connected to an auxiliary puncture device. Specifically: the main surgical robot hand, upon receiving a first puncture operation command, sends the command to the first processor and sends a prompt message to the auxiliary puncture device indicating that a puncture operation is to be performed; the first processor, upon receiving the first puncture operation command, sends a third movement command corresponding to the first puncture operation command to the secondary surgical robot hand; the auxiliary puncture device, upon receiving the prompt message, sends a scanning command to a medical imaging device; the medical imaging device, upon receiving the scanning command, performs a scanning operation on the puncture object; and the secondary surgical robot hand, according to the third movement command, moves the puncture needle to perform a puncture operation corresponding to the first puncture operation command. The puncture operation includes an orientation adjustment operation or a needle insertion operation. The orientation adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

[0025] In one possible implementation of the second aspect, the surgical robot includes: a main surgical robot hand, a first processor, and a secondary surgical robot hand. A puncture needle is disposed on the secondary surgical robot hand, and the first processor is communicatively connected to an auxiliary puncture device. Specifically: the main surgical robot hand is used to send a second puncture operation command to the first processor after receiving it; the first processor is used to determine a fourth movement command corresponding to the second puncture operation command; the first processor is used to send the fourth movement command to the secondary surgical robot hand and send a prompt message to the auxiliary puncture device indicating that a puncture operation is to be performed; the auxiliary puncture device is used to send a scanning command to a medical imaging device after receiving the prompt message; the medical imaging device is used to perform a scanning operation on the puncture object after receiving the scanning command; the secondary surgical robot hand is used to move the puncture needle according to the fourth movement command to perform a puncture operation corresponding to the second puncture operation command. The puncture operation includes an orientation adjustment operation or a needle insertion operation. The orientation adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

[0026] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps performed by the processor in the assisted puncture device as described in any of the first aspects above.

[0027] Fourthly, embodiments of this application provide a computer program product that, when run on a server, causes the server to execute the steps performed by the processor in the assisted puncture device described in any of the first aspects above.

[0028] Fifthly, embodiments of this application provide a chip, including: a processor for calling and running a computer program from a memory, causing an electronic device on which the chip is installed to perform the steps performed by the processor in the assisted puncture device as described in any of the first aspects above.

[0029] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect above, and will not be repeated here. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of the structure of a puncture system provided in one embodiment of this application;

[0032] Figure 2 This is a schematic diagram of the puncture procedure provided in one embodiment of this application;

[0033] Figure 3 This is a schematic diagram of tissue segmentation and display during the preoperative imaging scanning stage in one embodiment of this application;

[0034] Figure 4 This is a schematic diagram of the initial puncture path displayed on a CT image provided in one embodiment of this application;

[0035] Figure 5 This is an interactive diagram of an example puncture method provided in one embodiment of this application;

[0036] Figure 6 This is a schematic diagram of the interface between the needle tip and the predicted target point in the same tomographic image in an embodiment of the puncture method provided in this application.

[0037] Figure 7 This is a schematic diagram of the interface between the needle tip and the predicted target point located in different tomographic images in an embodiment of the puncture method provided in this application.

[0038] Figure 8 This is a schematic diagram of the interface after extracting the needle path outline in an example of a puncture method provided in one embodiment of this application.

[0039] Figure 9 This is a schematic diagram of the interface for same-layer posture adjustment in an example of a puncture method provided in one embodiment of this application;

[0040] Figure 10This is a schematic diagram of the interface for cross-layer posture adjustment in an example of a puncture method provided in one embodiment of this application;

[0041] Figure 11 This is a schematic diagram of the structure of an auxiliary puncture device provided in one embodiment of this application;

[0042] Figure 12 This is a schematic diagram of the structure of a surgical robot provided in one embodiment of this application. Detailed Implementation

[0043] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0044] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0045] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0046] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0047] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0048] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0049] Current master-slave real-time guided puncture protocols require physicians to rely on clinical experience to judge the puncture path and outcome after position adjustment. Because these protocols depend heavily on the physician's subjective medical experience, it's difficult to guarantee that the needle will reach the optimal puncture point when the physician lacks experience. This leads to repeated position adjustments, imaging scans, and punctures during the procedure. These repeated adjustments, scans, and punctures increase patient discomfort and radiation exposure, prolong the procedure, and increase surgical risks.

[0050] To address the aforementioned technical problems, this application provides an auxiliary puncture device, a puncture system, and a computer-readable storage medium. The auxiliary puncture device acquires multiple tomographic images obtained by a medical imaging device scanning the puncture target; and acquires puncture status information of the puncture needle in a surgical robot, including the needle's status information during the scanning operation of the medical imaging device. Based on the puncture status information, it determines the position information of the needle tip and the position information of a predicted target point, where the predicted target point is the spatial point reached by the needle tip after the needle is inserted according to the posture in the puncture status information. Then, based on the position information of the needle tip and the predicted target point, the needle tip and the predicted target point are marked on a first tomographic image to obtain a marked first tomographic image, which includes at least one or two of the multiple tomographic images. The marked first tomographic image is displayed through a display module. By annotating the needle tip and the predicted target on the first tomographic image and displaying the annotated image, even inexperienced doctors can determine the location of the needle tip and the predicted target point that the needle tip can reach based on the displayed information. Therefore, the assisted puncture device in this embodiment can assist doctors in determining the needle tip and the puncture point that the needle can reach, reducing reliance on the doctor's experience, improving puncture accuracy, and avoiding repeated adjustments, image scanning, and punctures during the procedure.

[0051] The auxiliary puncture device, puncture system, and computer-readable storage medium provided in this application are described below with reference to specific embodiments.

[0052] See Figure 1 This is a schematic diagram of a puncture system provided in one embodiment of this application. Figure 1 As shown, the puncture system in this embodiment includes: an auxiliary puncture device, a surgical robot, and a medical imaging device. The auxiliary puncture device is communicatively connected to both the surgical robot and the medical imaging device, and includes a display module.

[0053] Medical imaging equipment is used to scan the puncture target, obtain multiple tomographic images, and send these images to an auxiliary puncture device.

[0054] The surgical robot is used to send the puncture status information of the puncture needle in the surgical robot to the auxiliary puncture device. The puncture status information includes the status information of the puncture needle when the medical imaging device performs the scanning operation on the puncture object.

[0055] An auxiliary puncture device is used to determine the position information of the needle tip and the predicted target point based on the puncture status information. The predicted target point is the spatial point reached by the needle tip after the puncture needle completes insertion according to the posture in the puncture status information. Then, based on the position information of the needle tip and the position information of the predicted target point, the needle tip and the predicted target point are marked on the first tomographic image to obtain the marked first tomographic image. The first tomographic image includes at least one or two tomographic images. Finally, the marked first tomographic image is displayed through a display module.

[0056] The puncture system in this application embodiment marks the needle tip and the predicted target on the tomographic image and displays the marked tomographic image, so that the doctor has a clear understanding of the position of the needle tip and the position that the needle tip can reach (i.e., the position of the predicted target). This helps the doctor to judge the needle tip and the puncture point that the puncture needle can reach, reduces the dependence on the doctor's experience, and improves the accuracy of puncture.

[0057] like Figure 1 As shown, the surgical robot in this embodiment may include a surgical robot master hand, a first processor, and a surgical robot slave hand, which are connected in series via communication. It is understood that in the surgical robot, the surgical robot master hand is the operating end, and the surgical robot slave hand is the executing end. The user remotely controls the surgical robot slave hand to perform puncture operations by operating the surgical robot master hand.

[0058] like Figure 1As shown, in order to protect doctors from radiation during medical imaging scanning, the puncture system is divided into two parts: outside the operating room and inside the operating room. The auxiliary puncture equipment and the main hand of the surgical robot, which are operated by the doctor, are located outside the operating room; inside the operating room are: medical imaging equipment for scanning the puncture object, and the secondary hand of the surgical robot for puncturing the object.

[0059] In some embodiments, the surgical robot hand may include a robotic arm, a puncture tip attached to the end of the robotic arm, and a puncture needle mounted on the puncture tip. During a puncture procedure, the robotic arm of the surgical robot hand moves the puncture needle via the puncture tip, thereby achieving the puncture operation on the target object. Generally, the puncture operation of the puncture needle may include an orientation adjustment operation and a needle insertion operation, wherein: the orientation adjustment operation includes adjusting the posture of the puncture needle, and the needle insertion operation includes adjusting the puncture depth of the puncture needle.

[0060] For example, the first processor is used to convert the operation instructions applied by the doctor to the main hand of the surgical robot into movement instructions that can be executed by the slave hand of the surgical robot, and send the movement instructions to the slave hand of the surgical robot. The slave hand of the surgical robot executes the movement instructions to move the puncture needle, thereby completing the puncture operation.

[0061] Understandably, in situations like Figure 1 In the illustrated embodiment, the first processor is located inside the operating room. In other embodiments, the first processor may be located outside the operating room, or the first processor may be integrated with the surgical robot's slave hand or master hand. The specific location and form of the first processor in this application are not specifically limited.

[0062] It should be understood that medical imaging equipment is a medical scanning device capable of obtaining tomographic images, such as CT scanning equipment, MR (Magnetic Resonance) scanning equipment, PET / MR (Positron Emission Tomography / Magnetic Resonance) scanning equipment, PET-CT (Positron Emission Tomography-Computed Tomography) scanning equipment, or ultrasound scanning equipment. This application does not limit the specific type of medical imaging equipment, as long as the medical imaging equipment can obtain multiple tomographic images that can guide the puncture process.

[0063] It is understood that the communication connection in the embodiments of this application can be a wireless communication connection. For example, a wireless communication connection can be implemented using wireless communication technologies such as Bluetooth (BT), Wireless-Fidelity (WiFi), or Near Field Communication (NFC). Of course, the communication connection in the embodiments of this application can also be a wired communication connection, and this application does not limit or elaborate on this.

[0064] In some embodiments, the scanning operation of the medical imaging device on the puncture object can be manually controlled by the doctor. For example, when the doctor is operating the main hand of the surgical robot to perform a puncture operation, he can activate the medical imaging device to perform the scanning operation by stepping on the control pedal of the medical imaging device.

[0065] In some other embodiments, the scanning operation of the medical imaging equipment can be automatically controlled by the puncture-assisted device.

[0066] For example, when a doctor is operating a surgical robot to perform a puncture, the main hand of the surgical robot, upon receiving the first puncture operation command, sends the command to the first processor and a prompt message to the auxiliary puncture device indicating that the puncture operation is about to begin. Upon receiving the first puncture operation command, the first processor sends a third movement command corresponding to the first puncture operation command to the secondary hand of the robot. Upon receiving the prompt message, the auxiliary puncture device sends a scanning command to the medical imaging equipment. Upon receiving the scanning command, the medical imaging equipment scans the puncture target. The secondary hand of the surgical robot moves the puncture needle according to the third movement command, performing the puncture operation corresponding to the first puncture operation command. The puncture operation includes posture adjustment or needle insertion. Posture adjustment is used to adjust the posture of the puncture needle, and needle insertion is used to adjust the puncture depth. It can be understood that the first puncture operation command can be the doctor's operation on the main hand of the surgical robot, the prompt message can be a notification sent by the main hand of the surgical robot to the auxiliary puncture device indicating that the puncture operation is about to begin, and the puncture target can be the patient. In this embodiment, upon receiving the first puncture operation command, the main hand of the surgical robot prompts the auxiliary puncture device to control the medical imaging equipment for scanning. This allows for automatic control of the medical imaging equipment during real-time exposure scanning during the puncture operation, ensuring that the medical imaging equipment performs a scan every time the main hand receives a puncture operation command, thus guaranteeing the timeliness of the medical imaging scan. This embodiment utilizes the time interval between the main hand receiving the puncture operation command and the surgical robot executing the command to perform a medical imaging scan. This allows for medical imaging scanning before the puncture operation is performed, avoiding blind punctures caused by performing the puncture but not conducting a medical imaging scan.

[0067] It is understandable that the specific function of the third movement command is related to the puncture operation corresponding to the first puncture operation command. When the puncture operation corresponding to the first puncture operation command is an attitude adjustment operation, the third movement command is used to control the surgical robot to move the puncture needle for attitude adjustment; when the puncture operation corresponding to the first puncture operation command is a needle insertion operation, the third movement command is used to control the surgical robot to move the puncture needle for needle insertion.

[0068] For example, when a doctor is operating a surgical robot to perform a puncture, after receiving a second puncture operation command, the main hand of the surgical robot sends the second puncture operation command to the first processor. The first processor determines the fourth movement command corresponding to the second puncture command, and then sends the fourth movement command to the secondary hand of the surgical robot, and sends a prompt message to the auxiliary puncture device to perform the puncture operation. After receiving the prompt message, the auxiliary puncture device sends a scanning command to the medical imaging device. After receiving the scanning command, the medical imaging device performs a scanning operation on the puncture object. The secondary hand of the surgical robot moves the puncture needle according to the fourth movement command to perform the puncture operation corresponding to the second puncture operation command. The puncture operation includes posture adjustment or needle insertion. Posture adjustment is used to adjust the posture of the puncture needle, and needle insertion is used to adjust the puncture depth. It can be understood that the second puncture operation command can be the doctor's operation on the main hand of the surgical robot, the prompt message can be the notification from the first processor to the auxiliary puncture device that the surgical robot is about to perform a puncture operation, and the puncture object can be the patient. In this embodiment, after receiving the second puncture operation command, the main hand of the surgical robot sends the command to the first processor. Upon receiving the command, the first processor prompts the auxiliary puncture device to control the medical imaging equipment for scanning. This allows for automatic real-time exposure control of the medical imaging equipment during the puncture operation, ensuring that the medical imaging equipment performs a scan every time the main hand receives a puncture command, guaranteeing timely medical imaging. In this embodiment, the time interval between the first processor sending the fourth movement command corresponding to the second puncture operation command to the surgical robot and the robot executing the fourth movement command is used for medical imaging scanning. This allows for medical imaging scanning before the puncture operation, avoiding blind punctures caused by performing the puncture but not performing a medical imaging scan. Furthermore, sending the instruction information from the first processor to the auxiliary puncture device shortens the scanning time compared to sending the instruction information from the main hand of the surgical robot, thus reducing the radiation dose to the puncture recipient.

[0069] It is understandable that the specific function of the fourth movement command is related to the puncture operation corresponding to the second puncture operation command. When the puncture operation corresponding to the second puncture operation command is an attitude adjustment operation, the fourth movement command is used to control the surgical robot to move the puncture needle for attitude adjustment; when the puncture operation corresponding to the second puncture operation command is a needle insertion operation, the fourth movement command is used to control the surgical robot to move the puncture needle for needle insertion.

[0070] It should be understood that the first puncture operation instruction and the second puncture operation instruction may be the same or different, and this application does not impose any restrictions on this.

[0071] After introducing the puncture system, the following section will combine... Figure 1 The puncture system shown is provided as an example to illustrate the puncture process based on the puncture system in the embodiments of this application.

[0072] Figure 2 This is a schematic diagram of a puncture procedure provided in one embodiment of this application. Figure 2 As shown, the puncture process mainly includes steps S201 to S206, and each step is explained below.

[0073] S201, Preoperative imaging scan.

[0074] Specifically, before the biopsy begins, the affected area of ​​the patient is scanned using medical imaging equipment. For example, when the medical imaging equipment is a CT scanner, multi-phase contrast-enhanced scanning and plain scanning can be performed. The images obtained from the scan can clearly define the nature, outline, and extent of the lesion, and can also enable analysis of vital organs.

[0075] For example, during the preoperative scanning phase, the abdominal liver and thoracic lung lobes can be reconstructed in three dimensions in the transverse coronal sagittal orientation and displayed using VR (Virtual Reality).

[0076] exist Figure 2 In the illustrated embodiment, the tumor site is used as the target location for puncture, and operations such as target organ segmentation, tumor site extraction, and dangerous tissue identification can be achieved during the preoperative imaging scanning stage.

[0077] For example, during the preoperative imaging scanning stage, the contours of important tissues and organs such as lesions, soft tissues, and target organs can be extracted based on the DICOM images obtained from the CT scanning equipment, and the extracted images can be displayed through the display module.

[0078] For example, Figure 3 This is a schematic diagram of tissue segmentation obtained during the preoperative imaging scanning stage in one embodiment of this application. Figure 3As shown, this embodiment uses a CT scanning device for scanning, where the lesion is located on the liver. Figure 3 The outline of lesion 301 is marked in the image. In this embodiment, marking the lesion in the image during the preoperative imaging stage can help doctors locate the lesion and understand the surrounding conditions.

[0079] S202, Initial puncture path planning.

[0080] Based on the tissue model obtained during the preoperative imaging scan stage, i.e. Figure 3 The tissue segmentation diagram shown automatically plans the initial puncture path while avoiding important tissues and organs such as blood vessels, nerves, and bones.

[0081] The initial puncture path may include the following details: the puncture point on the body surface, the target point, the needle insertion angle, and the needle insertion depth. For example, Figure 4 This is a schematic diagram of the initial puncture path displayed on an image obtained by a CT scanning device, provided in one embodiment of this application. Figure 4 As shown, in this embodiment, the initial puncture path includes a surface puncture point 401, a puncture target point 402, and a needle path 403 connecting the surface puncture point 401 and the puncture target point 402, wherein the puncture target point 402 is located within the lesion outline 404.

[0082] In some embodiments, the initial puncture path can also be displayed on a 3D image, wherein the conversion from a 2D image to a 3D image can be achieved by a 3D reconstruction method, which will not be elaborated in this application.

[0083] S203, Intraoperative navigation and positioning.

[0084] A distal end-effector is attached to the surgical robot's hand and fixed next to the medical imaging equipment. Then, the surgical robot's navigation system determines the puncture target space, which can also be called the patient space, the surgical robot's hand space, and the medical imaging space of the medical imaging equipment. The registration relationship between these three spaces is obtained, resulting in a registration matrix. Coordinate transformations between the three spaces can be achieved using the registration rectangle.

[0085] S204, Intraoperative puncture preparation.

[0086] Based on the initial puncture path planning, the doctor selects a suitable length of puncture needle and installs it on the puncture end of the surgical robot's hand. After the puncture needle is installed, the surgical robot automatically controls the robotic arm to move the puncture needle to the puncture starting point, which is the surface puncture point in the initial path planning. The doctor then adjusts the needle's insertion angle to match the angle planned in the initial puncture path.

[0087] S205, intraoperative puncture.

[0088] like Figure 2 As shown, the intraoperative puncture stage mainly includes three parts: needle tract posture adjustment, needle insertion, and real-time exposure. Among them, needle tract posture adjustment mainly includes adjusting the posture of the puncture needle, while needle insertion mainly includes adjusting the puncture depth of the puncture needle; real-time exposure mainly includes scanning the puncture object with medical images during posture adjustment and needle insertion.

[0089] In this embodiment, the physician can choose between automatic or manual posture adjustment based on the clinical situation. Automatic posture adjustment refers to the auxiliary puncture device automatically controlling the surgical robot to adjust the posture of the puncture needle. Manual posture adjustment refers to the physician manipulating the surgical robot to adjust the posture of the puncture needle by translation, rotation, etc.

[0090] In this embodiment of the application, during the needle tract adjustment and needle insertion process, real-time exposure is performed by manipulating medical imaging equipment, and the exposure range is set within the needle insertion range, that is, the exposure range is set within the tomographic range where the needle tract is located. In this way, the movement direction of the needle in the patient's body and the actual position reached by the needle tip are observed in real time, thereby judging the current puncture situation and avoiding the impact of puncture on the patient's important tissues and organs, where the patient is the puncture subject.

[0091] S206, puncture completed.

[0092] After the needle is inserted through the puncture tract, the tip of the puncture needle reaches the lesion area, completing the puncture procedure.

[0093] It is understood that the lesion area in the embodiments of this application is a broad concept, and the definition of the lesion area may vary depending on the type of surgery. For example, for puncture surgeries such as biopsy and injection, the lesion area refers to the area surrounded by the lesion outline; for puncture surgeries such as ablation and particle implantation, the lesion area includes the area surrounded by the lesion outline, as well as the area within a certain range of the lesion edge or periphery.

[0094] It is understandable that the intraoperative puncture stage is the most critical and important stage in the above-mentioned puncture process. The puncture method in this application embodiment mainly improves the intraoperative puncture stage. The puncture method in this application embodiment will be described below with reference to the accompanying drawings.

[0095] See Figure 5 This is an interactive diagram illustrating an example of a puncture method provided in this application. This method is applicable to... Figure 1 The puncture system shown below, in conjunction with Figure 5 The puncture method in the embodiments of this application will be described. For example... Figure 5 As shown, the puncture method includes: S501 to S506.

[0096] S501, the medical imaging equipment scans the puncture object to obtain multiple tomographic images.

[0097] S502, the medical imaging equipment sends multiple tomographic images to the assisted puncture device.

[0098] In some embodiments, the scanning operation may be in Figure 2 The image shows a scanning procedure performed prior to intraoperative puncture. For example, it could be a scanning procedure performed during the preoperative imaging phase.

[0099] In other embodiments, the scanning operation can be performed during posture adjustment or during needle insertion. The posture adjustment process includes adjusting the orientation of the puncture needle, and the needle insertion process includes adjusting the puncture depth. Performing the scanning operation during posture adjustment or needle insertion ensures that the scanning operation is carried out during the main stage of the puncture, allowing the user to monitor the puncture process in real time through tomographic images, reducing the difficulty of puncture and improving puncture accuracy.

[0100] In some embodiments, during the posture adjustment process, the scanning device performs a scanning operation before the surgical robot performs the posture adjustment operation; during the needle insertion process, the scanning device performs a scanning operation before the surgical robot performs the needle insertion operation. By scanning before the surgical robot performs the posture adjustment operation or the needle insertion operation, the guiding role of the scanning on the posture adjustment operation or the needle insertion operation can be ensured, avoiding blind posture adjustment or needle insertion.

[0101] S503, the surgical robot sends the puncture status information of the puncture needle in the surgical robot to the auxiliary puncture device. The puncture status information includes the status information of the puncture needle when the medical imaging device scans the puncture object.

[0102] For example, the puncture status information of the puncture needle may include at least one of: puncture depth, puncture speed, and posture angle, wherein the puncture depth represents the depth to which the tip of the puncture needle enters the skin, the posture angle represents the angle of the puncture needle relative to the punctured object, and the puncture speed represents the speed at which the puncture needle moves along the insertion direction. Alternatively, the puncture speed of the puncture needle can also be understood as the speed at which the puncture needle moves along its length.

[0103] In some embodiments, the puncture assist device displays puncture status information via a display module. For example, the puncture assist device displays at least one of puncture depth, puncture speed, and posture angle via the display module. By displaying puncture status information, the user can see the puncture status of the puncture needle at any time, thereby improving puncture accuracy.

[0104] It should be understood that puncture status information can be theoretical data for the surgical robot when performing puncture operations. In this case, puncture status information can be obtained through theoretical calculations. For example, the puncture status information can be determined based on the theoretical data on which the surgical robot performs puncture operations, which will not be elaborated here.

[0105] In addition, puncture status information can also be the actual status information of the surgical robot when performing puncture operations. In this case, puncture status information can be obtained by real-time tracking and identification of the actual position of the puncture needle. For example, image analysis can be performed on the tomographic images obtained during the puncture process to track and identify the actual position of the puncture needle and obtain the actual position of the puncture needle. The puncture speed is determined based on the actual position information of the puncture needle. The puncture depth and attitude angle are determined by comparing the actual position of the puncture needle with the position information of the puncture object. These details will not be elaborated here.

[0106] S504, the auxiliary puncture device determines the position information of the needle tip and the position information of the predicted target point based on the puncture status information. The predicted target point is the spatial point reached by the needle tip after the puncture needle completes the insertion according to the posture in the puncture status information.

[0107] In some embodiments, the puncture status information may further include the position information of the needle tip, which is calculated based on the relative positional relationship between the puncture needle and the surgical robot; that is, the position information of the needle tip is a calculated theoretical value. When the puncture needle is in a normal state, the calculated position information of the needle tip is accurate. In this case, the auxiliary puncture device directly obtains the position information of the needle tip from the puncture status information. It is understood that a normal puncture needle state can mean that the puncture needle is not bent, broken, or otherwise abnormal.

[0108] For example, the position information of the needle tip may include the coordinates of the needle tip, which can be obtained by calculating the joint angles of the robotic arm. The specific calculation process will not be elaborated here.

[0109] In other embodiments, in addition to obtaining the coordinates of the needle tip directly from the surgical robot, the auxiliary puncture device can also obtain relevant parameters from the surgical robot and calculate the joint angle of the robotic arm based on the relevant parameters to obtain the coordinates of the needle tip.

[0110] In this embodiment of the application, the predicted target point is a prediction of the spatial point that the puncture needle may reach. For example, the position of the predicted target point includes the coordinates of the predicted target point. Specifically, the coordinates of the predicted target point can be obtained based on the initial coordinates of the needle tip in the robot coordinate system and the length of the puncture needle. The specific calculation process will not be elaborated here.

[0111] S505, the assisted puncture device marks the needle tip and the predicted target point on the first tomographic image according to the position information of the needle tip and the position information of the predicted target point, and obtains the marked first tomographic image. The first tomographic image includes at least one or two of the plurality of tomographic images.

[0112] For example, the needle tip is marked on the first tomographic image. Specifically, after determining the tomographic image where the needle tip is located, the coordinates of the needle tip are transformed from the robot coordinate system to the image coordinate system of the multiple tomographic images obtained by the medical imaging equipment, and then the needle tip is marked on the corresponding tomographic image. The specific coordinate transformation process will not be described in detail.

[0113] It should be understood that the needle tip and the predicted target each occupy a certain three-dimensional space. The needle tip can be located in one or more layers of the puncture object, so the tomographic images involving the needle tip can include one or more; similarly, the predicted target can also be located in one or more layers of the puncture object, so the tomographic images involving the predicted target can include one or more.

[0114] For example, when the needle tip and the predicted target are located in the same slice of the puncture object, the needle tip and the predicted target can be labeled on the same slice image. In this case, the first slice image includes one of the multiple slice images. When the needle tip and the predicted target are located in different slices of the puncture object, the needle tip and the predicted target are labeled on two different slice images. In this case, the first slice image includes at least two of the multiple slice images.

[0115] S506, the auxiliary puncture device displays the first tomographic image after annotation through the display module in the auxiliary puncture device.

[0116] It is understood that the first fault image displayed in the display module after annotation can be a partial fault image in the first fault image or all fault images in the first fault image; this application does not limit this.

[0117] For example, the display module can display the tomographic image where the center of the needle tip is located in all tomographic images marked with needle tips; and the display module can display the tomographic image where the center of the predicted target point is located in all tomographic images marked with predicted target points.

[0118] In some embodiments, when the needle tip and the predicted target are located in the same sectional view of the puncture object, the needle tip and the predicted target will be marked on the same sectional image. In this case, the sectional image displayed by the display module can be a sectional image marked with the needle tip and the predicted target.

[0119] For example, such as Figure 6 The image shown is a tomographic image displayed by the display module in an auxiliary puncture device according to an example of this application. Figure 6 As shown, the medical imaging device in this embodiment is a CT scanning device. In this embodiment, a total of 4 different tomographic images are displayed, in which the needle tip 601 and the predicted target point 602 are located on the same tomographic image.

[0120] For example, when the needle tip and the predicted target point are annotated on the same tomographic image, the first tomographic image after annotation also displays a line connecting the needle tip and the predicted target point. For example, in Figure 6 The display also shows a line 603 connecting the needle tip 601 and the predicted target point 602. Line 603 indicates the direction of the puncture path when the puncture needle is inserted in the current puncture state, allowing the user to more clearly determine the location of the puncture path and thus more accurately determine whether to insert the puncture based on the current puncture state.

[0121] Understandably, if Figure 6 The image shown is a tomographic image of the needle insertion state. The needle tip 601 and the predicted target point 602 are located on the same tomographic image, therefore this needle insertion process is a same-layer insertion. At this time, both the needle tip and the predicted target point are on the -729.125mm layer. The two marked points are connected by a line 603, which can be a dashed line to indicate the subsequent needle insertion path, facilitating the doctor's judgment of the tissue information that will be traversed during the subsequent puncture. As the needle depth increases, the needle tip 601 will continuously approach the predicted target point 602 along the dashed line until the puncture is completed.

[0122] In other embodiments, when the needle tip and the predicted target are located on two different tomographic images of the puncture site, the needle tip and the predicted target will be marked on two different tomographic images.

[0123] For example, such as Figure 7 The image shown is a tomographic image displayed by the display module in an auxiliary puncture device according to an example of this application. Specifically, as shown... Figure 7 As shown, the medical imaging device in this embodiment is a CT scanning device. In this embodiment, a total of 4 different tomographic images are displayed, of which the needle tip 701 and the predicted target point 702 are located on two different tomographic images.

[0124] Understandably, if Figure 7 The image shown is a tomographic image of the needle insertion state. The needle tip 701 and the predicted target point 702 are located on different tomographic images, therefore this needle insertion process is a cross-layer insertion. At this point, the real-time needle tip 701 is in the -729.125mm layer, and the predicted target point 702 is in the -727.875mm layer. As the needle insertion depth increases, the real-time needle tip 701 will puncture from the initial layer (i.e., the -729.125mm layer) to the target layer (i.e., the -727.875mm layer) until the puncture is completed.

[0125] Understandably, in Figure 6 and Figure 7 In order to facilitate user differentiation, different colors and shapes can be used to mark the needle tip and the predicted target. Of course, in some other embodiments, the same shape but different colors can be used to mark the needle tip and the predicted target, or the same color but different shapes can be used to mark the needle tip and the predicted target. This application does not limit the specific form of the marking.

[0126] In some embodiments, while displaying the annotated first tomographic image through the display module, adjacent tomographic images of the first tomographic image are also displayed through the display module. By displaying adjacent tomographic images, the movement of the needle tip and the predicted target point in different tomographic layers can be observed during puncture; simultaneously, when the needle tip and the predicted target point move onto adjacent tomographic images, they can be displayed quickly.

[0127] It is understood that adjacent fault images to the first fault image can be fault images corresponding to the fault preceding the first fault image; or they can be images corresponding to the fault following the first fault image. Furthermore, the fault images corresponding to the preceding fault image can include one or more, and the fault images corresponding to the following fault image can also include one or more, and this application does not impose any restrictions on this.

[0128] For example, in Figure 6 In the illustrated embodiment, in addition to the first tomographic image marked with needle tip 601 and predicted target 602, Figure 6 The image also shows three tomographic images adjacent to the first tomographic image.

[0129] For example, in Figure 7 In the illustrated embodiment, besides the two different first tomographic images labeled with needle tip 701 and predicted target 702, Figure 7 The image also shows two fault images adjacent to the first fault image, with each fault image being adjacent to one of the first fault images.

[0130] In some embodiments, the auxiliary puncture device displays puncture status information via a display module. The displayed puncture status information may include at least one of puncture depth, puncture speed, and posture angle, wherein puncture depth represents the depth to which the tip of the puncture needle enters the skin, posture angle represents the angle of the puncture needle relative to the puncture object, and puncture speed represents the speed at which the puncture needle moves along the insertion direction.

[0131] In some embodiments, when the puncture device determines that the puncture speed exceeds a certain threshold, it displays a warning message indicating that the puncture is too fast via a display module. By alerting users to excessively fast puncture speeds, the probability of premature puncture can be reduced, improving the safety of the puncture operation. For example, a warning message indicating that the puncture is too fast is displayed when the puncture speed exceeds 30 mm / s. The specific form of the warning message may be a pop-up window, or the way the puncture speed is displayed may change, such as displaying a warning icon along with the displayed puncture speed. This application does not limit the form of the warning message.

[0132] In other embodiments, the auxiliary puncture device may further: extract the needle path contour of the puncture needle based on multiple tomographic images, the needle path contour representing the three-dimensional structure of the puncture needle; determine the actual position information of the needle tip based on the needle path contour, the actual position information of the needle tip representing the position information of the needle tip within the needle path contour; if an abnormality is determined in the needle path contour based on the position information of the needle tip and the actual position information of the needle tip, a needle path abnormality warning is generated. In this embodiment, the needle path abnormality warning can promptly alert the user to abnormal information of the needle path, avoiding the impact of needle path abnormalities on puncture accuracy.

[0133] To facilitate understanding, the following is an exemplary description of the process of determining whether there are any abnormalities in the needle path contour based on the position information of the needle tip and the actual position information of the needle tip.

[0134] For example, the position information of the needle tip includes the theoretical coordinates of the needle tip, which are the coordinates of the needle tip reported by the surgical robot; the actual position information of the needle tip includes the actual coordinates of the needle tip, which are obtained based on the extracted needle path contour; since the actual coordinates of the needle tip are located in the image coordinate system, when determining whether there is an anomaly in the needle path contour, the actual coordinates of the needle tip are first transformed to the robot coordinate system to obtain the transformed actual coordinates of the needle tip; then, in the robot coordinate system, the transformed actual coordinates of the needle tip are compared with the theoretical coordinates of the needle tip. If the distance between the transformed actual coordinates of the needle tip and the theoretical coordinates of the needle tip is greater than a preset threshold, it is determined that there is an anomaly in the needle path contour; if the distance between the transformed actual coordinates of the needle tip and the theoretical coordinates of the needle tip is less than or equal to the preset threshold, it is determined that there is no anomaly in the needle path contour.

[0135] In some embodiments, when determining whether there is an anomaly in the needle path contour, the theoretical coordinates of the needle tip in the robot coordinate system can be transformed to the image coordinate system first to obtain the transformed theoretical coordinates of the needle tip; then, in the image coordinate system, the transformed theoretical coordinates of the needle tip are compared with the actual coordinates of the needle tip. If the distance between the transformed theoretical coordinates of the needle tip and the actual coordinates of the needle tip is greater than a preset threshold, it is determined that there is an anomaly in the needle path contour; if the distance between the transformed theoretical coordinates of the needle tip and the actual coordinates of the needle tip is less than or equal to the preset threshold, it is determined that there is no anomaly in the needle path contour.

[0136] For example, Figure 8 This is a schematic diagram of the needle path contour extracted in one embodiment of this application, as shown below. Figure 8 As shown, the medical imaging device in this embodiment is a CT scanning device. Figure 8 Figure a in the image is a tomographic image before needle path contour extraction. Figure 8 Image b in the diagram is a tomographic image after needle path contour extraction. Additionally, in... Figure 8 In Figure b, the target lesion is also marked, i.e. Figure 8 Figure b shows the needle path outline 801 and the target lesion 802.

[0137] It is understood that the needle path contour of the puncture needle can be obtained by three-dimensional reconstruction of the puncture needle based on multiple tomographic images, or it can be obtained by image recognition algorithm, which can be a machine learning algorithm; both three-dimensional reconstruction and image recognition algorithms can use common algorithms, and this application does not limit or elaborate on them.

[0138] For example, the needle tract abnormality warning generated by the auxiliary puncture device can be a warning displayed through a display module, such as displaying a needle tract abnormality prompt message on the display module, which can be a prompt window; or it can be a warning issued through other means, such as installing an alarm module in the auxiliary puncture device to issue an abnormality alarm. This application does not limit the specific form of the needle tract abnormality warning.

[0139] In other embodiments, the auxiliary puncture device can also annotate the target lesion of the puncture object. Specifically, the auxiliary puncture device annotates the target lesion of the puncture object on a second tomographic image among multiple tomographic images, obtaining an annotated second tomographic image. The second tomographic image includes tomographic images from the multiple tomographic images that scan to the target lesion; the annotated second tomographic image is displayed through a display module. In this embodiment, by annotating the target lesion, the user can see the lesion of the puncture object more clearly, thereby enabling the user to have more precise control over the puncture process and improving the user experience.

[0140] It is understandable that lesions are generally three-dimensional structures, so the target lesion of the puncture object may exist in multiple slices, that is, the target lesion may be scanned by multiple slice images.

[0141] In some embodiments, the second tomographic image includes only one tomographic image. In this case, the second tomographic image can be a tomographic image scanned to the center of the target lesion, or the second tomographic image can be the tomographic image with the largest lesion area, or the second tomographic image can be a tomographic image that meets other conditions. This application does not limit this.

[0142] In other embodiments, the second tomographic image may also include multiple tomographic images, or the second tomographic image may include all tomographic images scanned to the target lesion. This application does not impose any limitations on this.

[0143] In some embodiments, the target lesion is also marked on the tomographic image labeled with the predicted target point; that is, the first tomographic image labeled with the predicted target point and the second tomographic image labeled with the target lesion can be the same tomographic image. By marking the target lesion on the tomographic image where the predicted target point is located, it is convenient for the user to determine whether the target lesion can be reached by puncturing with the puncture needle in its current puncture state. For example, when the predicted target point coincides with the target lesion, or when the predicted target point is within the outline of the target lesion, it can be determined that the target lesion can be reached by puncturing with the puncture needle in its current puncture state. Otherwise, it is determined that the target lesion cannot be reached by puncturing with the puncture needle in its current puncture state.

[0144] In other embodiments, the needle tip, the predicted target, and the target lesion can be labeled on the same tomographic image. The first tomographic image labeled with the needle tip and the predicted target includes a tomographic image, and the second tomographic image labeled with the target lesion includes a tomographic image. The first tomographic image and the second tomographic image are the same tomographic image.

[0145] For example, the auxiliary puncture device displays the annotated second tomographic image through the display module, and also displays adjacent tomographic images of the second tomographic image through the display module. In this embodiment, the display of adjacent tomographic images of the second tomographic image is basically the same in method and technical effect as the display of adjacent tomographic images of the first tomographic image, and will not be described in detail here.

[0146] In some embodiments, the auxiliary puncture device further displays a three-dimensional image of the puncture needle and a three-dimensional image of the target lesion via a display module. The three-dimensional images of the puncture needle and the target lesion are obtained using multiple tomographic images, respectively. These images can be obtained through three-dimensional reconstruction of multiple tomographic images, or they can be generated using image recognition algorithms; this application does not impose any limitations on this method. In this embodiment, displaying three-dimensional images makes the presentation of the puncture needle and the target lesion more three-dimensional and vivid.

[0147] For example, the puncture needle and the target lesion are displayed in the same three-dimensional image. The relative positional relationship between the puncture needle and the target lesion in this three-dimensional image is the same as their relative positional relationship in the image coordinate system, where the image coordinate system is the coordinate system of multiple tomographic images. By displaying the three-dimensional images of the puncture needle and the target lesion in the same three-dimensional image and accurately presenting their relative positional relationship in the image coordinate system, the user can more intuitively see the puncture progress, thus facilitating the user's judgment on whether the puncture has been completed.

[0148] For example, in the same 3D image, the puncture needle and the target lesion are displayed using different colors. Using different colors to display the puncture needle and the target lesion makes it easier for users to distinguish between them and to better determine their relative positional relationship.

[0149] It should be understood that when the user determines, based on the image displayed by the display module, that the predicted target point does not coincide with the target lesion, or that the predicted target point is located outside the outline of the target lesion, an adjustment operation of the puncture needle is required. For ease of understanding, the adjustment process in the puncture method of this application embodiment will be exemplarily described below with reference to the accompanying drawings.

[0150] In some embodiments, the puncture system provides both automatic and manual posture adjustment methods, which the user can choose according to the clinical situation. The automatic and manual posture adjustment methods are described below by example.

[0151] When the user selects automatic posture adjustment, the auxiliary puncture device provides the adjustment data used for posture adjustment.

[0152] In some embodiments, the assisted puncture device may further: upon receiving a first posture adjustment command, identify the first location information of the target lesion from multiple tomographic images; obtain first adjustment data based on the first location information and puncture status information, wherein the first adjustment data represents the posture change data of the puncture needle when the predicted target point is adjusted to the first position corresponding to the first location information; and send a first movement command corresponding to the first adjustment data to the surgical robot, wherein the first movement command is used to control the surgical robot to move the puncture needle for posture adjustment; wherein the first posture adjustment command instructs the assisted puncture device to automatically adjust its posture, i.e., the user selects the automatic posture adjustment function. In this embodiment, the assisted puncture device obtains the location of the target lesion by performing image recognition on multiple tomographic images, wherein the location of the target lesion is the first location information, and obtains the first adjustment data based on the first location information and puncture status information. This method can achieve automatic posture adjustment, saving the user's posture adjustment time.

[0153] In some embodiments, before sending the first movement command corresponding to the first adjustment data to the surgical robot, the auxiliary puncture device displays the first adjustment data in a display module. If an instruction to use the first adjustment data is received, the first movement command corresponding to the first adjustment data is sent to the surgical robot. That is, in the automatic posture adjustment process of this embodiment, user confirmation is required before specific posture adjustment can be performed, which is equivalent to adding a doctor's review and confirmation step, improving the reliability of the puncture method and the accuracy of the puncture results.

[0154] In some embodiments, during automatic posture adjustment, the auxiliary puncture device can also determine whether needle withdrawal is necessary before posture adjustment. For example, before sending the first movement command corresponding to the first adjustment data to the surgical robot, the auxiliary puncture device performs the following steps: if the auxiliary puncture device determines that the puncture depth of the puncture needle is greater than a first threshold, or the distance between the predicted target point and the first position is greater than a second threshold, or direct posture adjustment based on the first adjustment data would cause interference between the puncture needle and the target tissue / organ, the auxiliary puncture device sends a second movement command to the surgical robot. The second movement command controls the surgical robot to move the puncture needle to withdraw the needle. After needle withdrawal is completed, the processor sends the first movement command corresponding to the first adjustment data to the surgical robot. The first movement command controls the surgical robot to move the puncture needle to adjust the posture.

[0155] Understandably, in the case of automatic posture adjustment, if the puncture depth of the needle exceeds the first threshold, it indicates that the needle has penetrated to a relatively deep depth, and direct posture adjustment may cause significant damage to the puncture target. If the distance between the predicted target point and the first position exceeds the second threshold, it indicates that the predicted target point deviates significantly from its target angle. In this case, if direct posture adjustment requires a large adjustment, it will cause significant damage to the puncture target. Interference between the needle and the target tissue / organ of the puncture target means that the needle may touch the target tissue / organ, potentially damaging it. Therefore, controlling the surgical robot to withdraw the needle before posture adjustment under these three conditions can significantly reduce damage to the puncture target and improve the safety of automatic posture adjustment. The target tissue / organ refers to important tissues / organs of the puncture target, such as blood vessels, bones, nerves, etc.

[0156] In some other embodiments, during automatic posture adjustment, before sending the first movement command corresponding to the first adjustment data to the surgical robot, the auxiliary puncture device performs the following steps: if the auxiliary puncture device determines that the puncture depth of the puncture needle is less than or equal to a first threshold, and predicts that the distance between the target point and the first position is less than or equal to a second threshold, and determines that direct posture adjustment based on the first adjustment data will not cause interference between the puncture needle and the target tissue or organ of the puncture object, the auxiliary puncture device sends the first movement command to the surgical robot, and the first movement command is used to control the surgical robot to move the puncture needle for posture adjustment.

[0157] Understandably, in the case of automatic posture adjustment, if the puncture depth of the puncture needle is less than or equal to the first threshold, it indicates that the puncture depth is shallow, and direct posture adjustment will cause less damage to the puncture subject. If the distance between the predicted target point and the first position is less than or equal to the second threshold, it indicates that the deviation angle of the predicted target point is small. In this case, the adjustment range of the puncture needle required for direct posture adjustment is small, and the damage to the puncture subject is also small. The puncture needle not interfering with the target tissue or organ of the puncture subject means that the puncture needle will not touch the target tissue or organ of the puncture subject. Therefore, when all three conditions are met, the surgical robot can be controlled to directly adjust its posture without withdrawing the needle.

[0158] For example, after determining that needle withdrawal is unnecessary and posture adjustment can be performed directly, the auxiliary puncture device displays the first adjustment data in the display module before sending the first movement command corresponding to the first adjustment data to the surgical robot. If an instruction to adopt the first adjustment data is received, the first movement command corresponding to the first adjustment data is sent to the surgical robot. That is, in the automatic posture adjustment process of this embodiment, user confirmation is required before specific posture adjustment can be performed, which is equivalent to adding a doctor's review and confirmation step, improving the reliability of the puncture method and the accuracy of the puncture results.

[0159] In some embodiments, during manual posture adjustment, the user controls the surgical robot to adjust the posture of the puncture needle. For example, the auxiliary puncture device can also: upon receiving a second posture adjustment command, send a manual posture adjustment command to the surgical robot. This manual posture adjustment command is an instruction from the surgical robot to control the puncture needle to adjust its posture in response to the user's operation; that is, in this embodiment, the user selects manual posture adjustment. This embodiment provides a manual posture adjustment process that meets the user's need for a high degree of autonomy, facilitating manual posture adjustment.

[0160] It is understood that in the specific pose adjustment process, pose adjustment can include: intra-slice pose adjustment, cross-slice pose adjustment, and free-mode pose adjustment. The following examples, with reference to the accompanying drawings, illustrate intra-slice pose adjustment, cross-slice pose adjustment, and free-mode pose adjustment. It is understood that the medical imaging equipment in the following examples is a CT scanning device.

[0161] (1) Adjust posture in the same layer.

[0162] Same-layer pose adjustment refers to the situation where the predicted target point before and after pose adjustment are on the same transverse section (i.e., on the same tomographic image). In this case, only the same-layer angle (i.e., the left and right tilt angle) needs to be changed, without changing the cross-layer angle (i.e., the head-to-foot tilt angle). Figure 9 As shown, in Figure 9 Figure a in the image is a tomographic image before attitude adjustment. Figure 9 Image b in the diagram is the tomographic image after orientation adjustment. For example... Figure 9 As shown in Figure a, the predicted target point 901 and the target lesion 902 are located on the same tomographic image before the pose adjustment, but they do not overlap. Therefore, pose adjustment in the same layer is required. After the angle in the same layer is changed, the position of the predicted target point changes accordingly on the Dicom image of the same transverse section (Z-axis coordinate -729.125mm). Figure 9 Image b in the figure is the tomographic image after pose adjustment, such as... Figure 9 As shown in Figure b, the predicted target point 903 coincides with the target lesion 902 after the orientation adjustment, thus completing the orientation adjustment.

[0163] In some embodiments, a same-layer orientation adjustment mode can be set. When the predicted target point and the target lesion are determined to be located in the same slice, the user selects the same-layer orientation adjustment mode during automatic or manual orientation adjustment. In this mode, only the angle within the same slice can be changed during orientation adjustment, and the angle across slices cannot be changed. By restricting the adjustment to only the angle within the same slice, the same-layer orientation adjustment mode can avoid the problem of accidentally crossing slices during orientation adjustment.

[0164] (2) Cross-layer posture adjustment.

[0165] Cross-layer pose adjustment refers to a phenomenon where the predicted target point before and after pose adjustment has a cross-layer sectional position (i.e., it is located on different tomographic images). The angle within the same layer (left and right deviation angle) may remain unchanged or change, while the angle across layers (head and foot deviation angle) changes. For example Figure 10 As shown, in Figure 10 Figure a in the image is a tomographic image before attitude adjustment. Figure 10 Image b in the diagram is the tomographic image after orientation adjustment. For example... Figure 10 As shown in Figure a, the predicted target point 1001 before orientation adjustment did not have a target lesion on the tomographic image. Therefore, it was necessary to adjust the predicted target point to a tomographic image showing the target lesion. In this case, cross-layer orientation adjustment was required. After the cross-layer angle was changed, the Z-axis coordinate of the predicted target point 1002 was updated from -729.125mm to -727.875mm, and the orientation-adjusted predicted target point 1002 coincided with the target lesion 1003, thus completing the orientation adjustment.

[0166] In some embodiments, a cross-layer orientation adjustment mode can be set. When the predicted target point and the target lesion are determined to be located in different slices, the user selects the cross-layer orientation adjustment mode during automatic or manual orientation adjustment. In this mode, only the cross-layer angle can be changed during orientation adjustment, and the same-layer angle cannot be changed. The cross-layer orientation adjustment mode allows the predicted target point to be quickly adjusted to the slice where the target lesion is located during orientation adjustment, thereby improving orientation adjustment efficiency.

[0167] Understandably, if the predicted target point and the target lesion still cannot be aligned after the predicted target point is adjusted to the slice where the target lesion is located, the same slice orientation adjustment mode can be selected, and the final orientation adjustment can be completed in the same slice orientation adjustment mode.

[0168] (3) Adjust your posture freely.

[0169] Free pose adjustment refers to the ability of the doctor to control the robotic arm to perform pose adjustment within the same layer and across layers simultaneously. After the needle path posture changes, the predicted target position can change both within the same layer and across layers. This will not be elaborated further here.

[0170] After introducing the posture adjustment process, the triggering process of the medical imaging device in the embodiments of this application will be described below.

[0171] In some embodiments, the puncture assist device automatically controls the medical imaging equipment to perform scanning operations.

[0172] For example, when the puncture assist device automatically controls the medical imaging equipment to perform scanning operations, the puncture assist device, upon receiving a prompt from the surgical robot to perform a puncture operation, controls the medical imaging equipment to execute the scanning operation. The puncture operation includes posture adjustment or needle insertion. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle. In this embodiment, the puncture assist device can automatically control the medical imaging equipment to perform scanning, which can improve the synchronization between the medical imaging equipment and the puncture, and improve the real-time performance of the scanning. A description of automatic scanning can be found in the preceding puncture system embodiment, and will not be repeated here.

[0173] After introducing the puncture method in the embodiments of this application, the structure of the auxiliary puncture device and surgical robot in the embodiments of this application will be described below with reference to the accompanying drawings.

[0174] See Figure 11 This is a schematic diagram of the structure of an auxiliary puncture device 11 provided in one embodiment of this application. Figure 11 As shown, the auxiliary puncture device in this embodiment includes a display module 1103 and a processor 1100. The processor 1100 is used to execute the steps performed by the auxiliary puncture device in any of the above puncture method embodiments.

[0175] In some other embodiments, the processor 1100 is configured to perform the following steps: acquire multiple tomographic images obtained by a medical imaging device scanning a puncture object; acquire puncture status information of a puncture needle in a surgical robot, the puncture status information including the status information of the puncture needle when the medical imaging device scans the puncture object; determine the position information of the needle tip and the position information of a predicted target point based on the puncture status information, the predicted target point being the spatial point reached by the needle tip after the puncture needle completes insertion according to the posture in the puncture status information; mark the needle tip and the predicted target point on a first tomographic image based on the position information of the needle tip and the position information of the predicted target point, thereby obtaining a marked first tomographic image; the first tomographic image includes at least one or two of the multiple tomographic images; and display the marked first tomographic image through the display module 1103.

[0176] It should be understood that the processor 1100 may acquire multiple tomographic images directly from the medical imaging equipment or indirectly from the medical imaging equipment; this application does not limit this. For example, the medical imaging equipment may directly report multiple tomographic images to the processor 1100, or the processor 1100 may actively acquire puncture status information from the medical imaging equipment; or after the medical imaging equipment obtains multiple tomographic images through scanning, it may send them to a relay device, from which the processor 1100 acquires the multiple tomographic images.

[0177] Similarly, the processor 1100 can obtain puncture status information directly from the surgical robot or indirectly from the surgical robot, and this application does not limit this. For example, the surgical robot can directly report puncture status information to the processor 1100, or the processor 1100 can actively obtain puncture status information from the surgical robot; or the surgical robot can also send the puncture status information to a relay device, and the processor 1100 obtains the puncture status information from the relay device.

[0178] For example, such as Figure 11 As shown, in some other embodiments, the auxiliary puncture device also includes a memory 1101 and a computer program 1102 stored in the memory 1101 and executable on the processor 1100. When the processor 1100 executes the computer program 1102, it implements the steps performed by the auxiliary puncture device in any of the above puncture method embodiments.

[0179] In some other embodiments, when the processor 1100 executes the computer program 1102, it performs the following steps: acquiring multiple tomographic images obtained by a medical imaging device scanning a puncture object; acquiring puncture status information of a puncture needle in a surgical robot, the puncture status information including the status information of the puncture needle when the medical imaging device scans the puncture object; determining the position information of the needle tip and the position information of a predicted target point based on the puncture status information, the predicted target point being the spatial point reached by the needle tip after the puncture needle completes insertion according to the posture in the puncture status information; marking the needle tip and the predicted target point on a first tomographic image based on the position information of the needle tip and the position information of the predicted target point, thereby obtaining a marked first tomographic image; the first tomographic image includes at least one or two of the multiple tomographic images; and displaying the marked first tomographic image through the display module 1103.

[0180] Figure 11 This is merely an example of the auxiliary puncture device 11 and does not constitute a limitation on the auxiliary puncture device 11. The auxiliary puncture device 11 may include more or fewer components than shown, or combine certain components, or different components.

[0181] The processor 1100 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0182] In some embodiments, the memory 1101 may be an internal storage unit of the assisted puncture device 11, such as a hard disk or memory of the assisted puncture device 11. In other embodiments, the memory 1101 may be an external storage device of the assisted puncture device 11, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the assisted puncture device 11. Further, the memory 1101 may include both internal storage units and external storage devices of the assisted puncture device 11. The memory 1101 is used to store operating systems, applications, boot loaders, data, and other programs, such as the program code of computer programs. The memory 1101 can also be used to temporarily store data that has been output or will be output.

[0183] See Figure 12 This is a schematic diagram of the structure of a surgical robot 12 provided in one embodiment of this application. Figure 12 As shown, the assisted puncture device in this embodiment includes a robot master hand 1203, a robot slave hand 1204, and at least one processor 1200. Figure 12 The diagram shows only one processor, a memory 1201, and a computer program 1202 stored in the memory 1201 and executable on the at least one processor 1200. When the processor 1200 executes the computer program 1202, it performs the steps executed by the first processor in any of the above embodiments of the puncture system.

[0184] Figure 12This is merely an example of surgical robot 12 and does not constitute a limitation on surgical robot 12. Surgical robot 12 may include more or fewer parts than shown, or combine certain parts, or different parts.

[0185] The processor 1200 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0186] In some embodiments, the memory 1201 may be an internal storage unit of the surgical robot 12, such as a hard drive or memory of the surgical robot 12. In other embodiments, the memory 1201 may be an external storage device of the surgical robot 12, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the surgical robot 12. Furthermore, the memory 1201 may include both internal and external storage units of the surgical robot 12. The memory 1201 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 1201 can also be used to temporarily store data that has been output or will be output.

[0187] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0188] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps performed by the auxiliary puncture device in the above-described puncture method embodiments, or the steps performed by the processor in the above-described auxiliary puncture device embodiments.

[0189] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to execute steps performed by the auxiliary puncture device in the above-described puncture method embodiments, or steps executed by the processor in the above-described auxiliary puncture device embodiments.

[0190] This application also provides a chip located in an electronic device, the chip including: a processing unit and a communication unit. The processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, pins, or circuits. The processing unit can execute computer instructions to cause the electronic device to perform the steps performed by the auxiliary puncture device in any of the puncture methods provided in this application, or the steps performed by the processor in the auxiliary puncture device embodiments described above.

[0191] Optionally, the computer instructions are stored in a storage unit.

[0192] Optionally, the storage unit can be an internal storage unit within the chip, such as a register or cache. Alternatively, it can be an external storage unit located within the terminal, such as a ROM or other types of static storage devices capable of storing static information and instructions, such as random access RAM. The processor mentioned above can be a CPU, microprocessor, ASIC, or one or more integrated circuits used to control the execution of a program for transmitting the aforementioned feedback information. The processing unit and the storage unit can be decoupled and located on different physical devices, connected via wired or wireless means to implement their respective functions, thereby supporting the system chip in implementing the various functions described in the above embodiments. Alternatively, the processing unit and the memory can also be coupled to the same device.

[0193] In this embodiment, the auxiliary puncture device, computer-readable storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0194] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a projection device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0195] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0196] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0197] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0198] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0199] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An auxiliary puncture device, characterized in that, The auxiliary puncture device includes a display module and a processor, the processor being used to perform the following steps: Acquire multiple tomographic images obtained by medical imaging equipment scanning the puncture target; Acquire puncture status information of the puncture needle in the surgical robot, the puncture status information including the status information of the puncture needle when the medical imaging device performs the scanning operation on the puncture object; Based on the puncture state information, the position information of the needle tip and the position information of the predicted target point are determined. The predicted target point is the spatial point reached by the needle tip after the puncture needle completes insertion according to the posture in the puncture state information. The position information of the needle tip is the theoretical value of the position information of the needle tip. Based on the location information of the needle tip and the location information of the predicted target, the needle tip and the predicted target are marked on the first tomographic image to obtain the marked first tomographic image. The first tomographic image includes at least one or two of the plurality of tomographic images. The first tomographic image after annotation is displayed through the display module; The puncture status information includes the position information of the needle tip, which is calculated based on the relative positional relationship between the puncture needle and the surgical robot. The processor further executes the following steps: Based on the multiple tomographic images, the needle track contour of the puncture needle is extracted, and the needle track contour represents the actual three-dimensional structure of the puncture needle; Based on the needle path contour, the actual position information of the needle tip of the puncture needle is determined. The actual position information of the needle tip represents the position information of the needle tip of the puncture needle in the needle path contour. The actual position information of the needle tip is: the position information of the actual location of the needle tip of the puncture needle. If an abnormality is determined in the needle path contour based on the position information of the needle tip and the actual position information of the needle tip, a needle path abnormality warning is generated.

2. The auxiliary puncture device according to claim 1, characterized in that, The scanning operation is performed during the posture adjustment process or during the needle insertion process. The posture adjustment process includes adjusting the posture of the puncture needle, and the needle insertion process includes adjusting the puncture depth of the puncture needle.

3. The auxiliary puncture device according to claim 1, characterized in that, When the needle tip and the predicted target are labeled on the same tomographic image, the first tomographic image after labeling also displays the line connecting the needle tip and the predicted target.

4. The auxiliary puncture device according to claim 1, characterized in that, The processor also performs the following steps: While displaying the annotated first fault image through the display module, the adjacent fault images of the first fault image are also displayed through the display module.

5. The auxiliary puncture device according to claim 1, characterized in that, The processor also performs the following steps: The display module also displays the puncture status information, which includes at least one of puncture depth, puncture speed, and posture angle. The puncture depth represents the depth to which the tip of the puncture needle enters the skin. The posture angle represents the angle of the puncture needle relative to the puncture object. The puncture speed represents the speed at which the puncture needle moves along the insertion direction.

6. The auxiliary puncture device according to any one of claims 1 to 5, characterized in that, The processor also performs the following steps: If a first posture adjustment command is received, the first location information of the target lesion of the puncture object is identified from the plurality of tomographic images, and the first posture adjustment command instructs the auxiliary puncture device to automatically adjust its posture. Based on the first position information and the puncture state information, first adjustment data is obtained. The first adjustment data represents the posture change data of the puncture needle when the predicted target point is adjusted to the first position corresponding to the first position information. A first movement command corresponding to the first adjustment data is sent to the surgical robot. The first movement command is used to control the surgical robot to move the puncture needle to adjust its position.

7. The auxiliary puncture device according to claim 6, characterized in that, Before sending the first movement command corresponding to the first adjustment data to the surgical robot, the processor performs the following steps: If the processor determines that the puncture depth of the puncture needle is greater than a first threshold, or the distance between the predicted target point and the first position is greater than a second threshold, or that direct posture adjustment based on the first adjustment data will cause the puncture needle to interfere with the target tissue or organ of the puncture object, then a second movement command is sent to the surgical robot. The second movement command is used to control the surgical robot to move the puncture needle to withdraw the needle.

8. The auxiliary puncture device according to claim 6, characterized in that, Before sending the first movement command corresponding to the first adjustment data to the surgical robot, the processor further performs the following steps: The first adjustment data is displayed through the display module; If an instruction to use the first adjustment data is received, a first movement instruction corresponding to the first adjustment data is sent to the surgical robot.

9. The auxiliary puncture device according to any one of claims 1 to 5, characterized in that, The processor also performs the following steps: If a second posture adjustment command is received, a manual posture adjustment command is sent to the surgical robot. The manual posture adjustment command is a command by which the surgical robot controls the puncture needle to adjust its posture in response to the user's operation. The second posture adjustment command instructs the user to select manual posture adjustment.

10. The auxiliary puncture device according to any one of claims 1 to 5, characterized in that, The scanning operation is performed by the auxiliary puncture device automatically controlling the medical imaging device. When the auxiliary puncture device automatically controls the medical imaging device to perform the scanning, the processor executes the following steps: If a prompt message is received from the surgical robot to perform a puncture operation, the medical imaging device is controlled to perform the scanning operation. The puncture operation includes an orientation adjustment operation or a needle insertion operation. The orientation adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

11. A puncture system, characterized in that, The puncture system includes: medical imaging equipment, a surgical robot, and an auxiliary puncture device as described in any one of claims 1 to 10, wherein: The medical imaging device is used to scan the puncture object, obtain multiple tomographic images, and send the multiple tomographic images to the auxiliary puncture device; The surgical robot is used to send the puncture status information of the puncture needle in the surgical robot to the auxiliary puncture device. The puncture status information includes the status information of the puncture needle when the medical imaging device performs the scanning operation on the puncture object. The auxiliary puncture device is used to determine the position information of the needle tip and the position information of the predicted target point based on the puncture state information. The predicted target point is the spatial point reached by the needle tip after the puncture needle completes insertion according to the posture in the puncture state information. The position information of the needle tip is the theoretical value of the position information of the needle tip when the puncture needle is in a normal state. The auxiliary puncture device is used to mark the needle tip and the predicted target on a first tomographic image based on the position information of the needle tip and the position information of the predicted target, thereby obtaining a marked first tomographic image; the first tomographic image includes at least one or two of the plurality of tomographic images. The auxiliary puncture device is used to display the annotated first tomographic image through the display module in the auxiliary puncture device; The puncture status information includes the position information of the needle tip, which is calculated based on the relative positional relationship between the puncture needle and the surgical robot. The auxiliary puncture device is also used to perform the following steps: Based on the multiple tomographic images, the needle track contour of the puncture needle is extracted, and the needle track contour represents the actual three-dimensional structure of the puncture needle; Based on the needle path contour, the actual position information of the needle tip of the puncture needle is determined. The actual position information of the needle tip represents the position information of the needle tip of the puncture needle in the needle path contour. The actual position information of the needle tip is: the position information of the actual location of the needle tip of the puncture needle. If an abnormality is determined in the needle path contour based on the position information of the needle tip and the actual position information of the needle tip, a needle path abnormality warning is generated.

12. The puncture system according to claim 11, characterized in that, The surgical robot includes: a main surgical robot hand, a first processor, and a secondary surgical robot hand. The puncture needle is disposed on the secondary surgical robot hand, and the main surgical robot hand is communicatively connected to the auxiliary puncture device; wherein: The main hand of the surgical robot is used to send the first puncture operation instruction to the first processor after receiving the first puncture operation instruction, and to send a prompt message to the auxiliary puncture device to perform the puncture operation. The first processor is configured to send a third movement command corresponding to the first puncture operation command to the surgical robot from the hand after receiving the first puncture operation command; The auxiliary puncture device is used to send a scanning command to the medical imaging device after receiving the prompt information; The medical imaging device is used to perform the scanning operation on the puncture object after receiving the scanning command; The surgical robot is used by hand to move the puncture needle according to the third movement command and perform a puncture operation corresponding to the first puncture operation command. The puncture operation includes a posture adjustment operation or a needle insertion operation. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

13. The puncture system according to claim 11, characterized in that, The surgical robot includes: a main surgical robot hand, a first processor, and a secondary surgical robot hand. The puncture needle is disposed on the secondary surgical robot hand, and the first processor is communicatively connected to the auxiliary puncture device. Wherein: The main hand of the surgical robot is used to send the second puncture operation command to the first processor after receiving the second puncture operation command; The first processor is configured to determine a fourth movement command corresponding to the second puncture operation command; The first processor is configured to send a fourth movement command from the hand to the surgical robot and to send a prompt message for performing a puncture operation to the auxiliary puncture device; The auxiliary puncture device is used to send a scanning command to the medical imaging device after receiving the prompt information; The medical imaging device is used to perform the scanning operation on the puncture object after receiving the scanning command; The surgical robot is used by hand to move the puncture needle according to the fourth movement command and perform a puncture operation corresponding to the second puncture operation command. The puncture operation includes a posture adjustment operation or a needle insertion operation. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it performs the steps performed by the processor in the assisted puncture device as described in any one of claims 1 to 10.

Images