Auxiliary puncture equipment, puncture system and computer readable storage medium

Abstract

The invention provides auxiliary puncture equipment, a puncture system and a computer readable storage medium, and the auxiliary puncture equipment comprises a display module, a processor and a memory, the processor is used for executing the following steps when calling the calculation program in the memory: acquiring a plurality of cross-sectional images obtained by scanning a puncture object by the medical imaging equipment; acquiring puncture state information of a puncture needle in the surgical robot; according to the puncture state information, position information of a needle point of the puncture needle and position information of a prediction target point are determined; according to the position information of the needle point and the position information of the prediction target, the needle point and the prediction target are marked on the first cross-sectional image, and the marked first cross-sectional image is obtained; and displaying the marked first cross-sectional image through a display module. By marking the needle point and the prediction target on the cross-sectional image for display, the dependency on doctor experience can be reduced.

Description

Technical Field

[0001] The present application belongs to the field of medical device technology, and in particular, relates to an auxiliary puncture device, a puncture system and a computer-readable storage medium. Background Art

[0002] Percutaneous puncture of lesions or soft tissues with surgical needles is an important part of modern clinical medicine and has attracted the attention of many researchers in recent years. Many clinical applications, such as biopsy, injection, neurosurgery and cancer treatment, use similar minimally invasive treatment methods.

[0003] The current puncture surgery solutions mainly include manual blind puncture, real-time image-guided puncture, master-slave real-time guided puncture, etc. Among them, master-slave real-time guided puncture means that the doctor controls the surgical robot outside the operating room (i.e., CT shielding room) to perform remote puncture, and controls the CT (Computed Tomography) equipment to perform real-time image scanning and display of the patient. In this solution, since the doctor performs remote operation outside the operating room, the doctor can avoid being harmed by radiation during the operation.

[0004] However, the master-slave real-time guided puncture scheme also has certain problems. During the puncture process, the doctor needs to rely on clinical experience to judge the puncture path and the puncture results after posture adjustment, and it is difficult to ensure that the puncture needle reaches the best puncture point after posture adjustment. That is, because the current master-slave real-time guided puncture scheme relies too much on the doctor's subjective medical experience, it is easy to cause the problem of inadequate puncture, which in turn leads to repeated posture adjustment, scanning and puncture during the operation, prolonging the operation time; multiple posture adjustment, scanning and puncture will also increase the patient's pain and radiation dose.

[0005] Therefore, how to reduce the dependence of the master-slave real-time guided puncture scheme on the doctor's experience, thereby improving the puncture accuracy of the master-slave real-time guided puncture scheme, has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The embodiments of the present application provide an auxiliary puncture device, a puncture system and a computer-readable storage medium, which can reduce the dependence of the master-slave real-time guided puncture scheme on the doctor's experience, thereby improving the puncture accuracy of the master-slave real-time guided puncture scheme.

[0007] In a first aspect, an embodiment of the present application provides an auxiliary puncture device, which includes: a display module and a processor, and the processor is used to perform the following steps: obtaining multiple tomographic images obtained by a medical imaging device performing a scanning operation on a puncture object; obtaining puncture status information of a puncture needle in a surgical robot, the puncture status information including status information of the puncture needle when the medical imaging device performs a scanning operation on the puncture object; determining position information of a needle tip point of the puncture needle and position information of a predicted target point based on the puncture status information, the predicted target point being: the spatial point reached by the needle tip point of the puncture needle after the puncture needle completes the insertion according to the current posture; marking the needle tip point and the predicted target point on a first tomographic image based on the position information of the needle tip point and the position information of the predicted target point, to obtain a marked first tomographic image, the first tomographic image including at least one or two tomographic images among the multiple tomographic images; and displaying the marked first tomographic image through the display module.

[0008] The auxiliary puncture device in the first aspect marks the needle tip point and the predicted target point on the tomographic image and displays the marked tomographic image, so that the doctor can clearly know the position of the needle tip point of the puncture needle and the position of the predicted target point that the needle tip point can reach. This can assist the doctor in judging the needle tip point of the puncture needle and the puncture point that the puncture needle can reach, reduce the dependence on the doctor's experience, and improve the accuracy of puncture.

[0009] In a possible implementation of the first aspect, the processor further performs the following steps: marking the target lesion of the puncture object on a second tomographic image among the multiple tomographic images to obtain a marked second tomographic image, the second tomographic image including the tomographic image of the target lesion scanned among the multiple tomographic images; and displaying the marked second tomographic image through a display module. In this implementation, by marking the target lesion, the user can see the lesion of the puncture object more clearly, thereby enabling the user to control the puncture process more accurately and improving the user experience.

[0010] In a possible implementation manner of the first aspect, the processor further performs the following steps: while displaying the annotated second tomographic image through the display module, also displaying adjacent tomographic images of the second tomographic image through the display module.

[0011] In a possible implementation of the first aspect, the processor further performs the following steps: displaying a three-dimensional image of the puncture needle and a three-dimensional image of the target lesion through a display module, wherein the three-dimensional image of the puncture needle and the three-dimensional image of the target lesion are obtained using a plurality of tomographic images, respectively. In this implementation, by displaying the three-dimensional image of the puncture needle and the three-dimensional image of the target lesion, the presentation of the puncture needle and the target lesion is made more three-dimensional and vivid.

[0012] In a possible implementation of the first aspect, the puncture needle and the target lesion are displayed in the same three-dimensional image, and the relative position relationship between the puncture needle and the target lesion in the same three-dimensional image is the same as the relative position relationship between the puncture needle and the target lesion in the image coordinate system, and the image coordinate system is the coordinate system where the multiple tomographic images are located. In this implementation, by displaying the puncture needle and the target lesion in the same three-dimensional image, and presenting the relative position relationship between the puncture needle and the target lesion truthfully, the user can more intuitively see the puncture progress, thereby facilitating the user to judge whether the puncture is completed.

[0013] In a possible implementation of the first aspect, the puncture state information includes the position information of the needle tip point, which is calculated based on the relative position relationship between the puncture needle and the surgical robot, and the processor further performs the following steps: extracting the needle track contour of the puncture needle based on multiple tomographic images, the needle track contour represents the three-dimensional structure of the puncture needle; determining the actual position information of the needle tip point of the puncture needle based on the needle track contour, the actual position information of the needle tip point represents the position information of the needle tip point of the puncture needle in the needle track contour; if it is determined that the needle track contour is abnormal based on the position information of the needle tip point and the actual position information of the needle tip point, a needle track abnormality warning is generated. In this implementation, the needle track abnormality warning can prompt the user with the abnormal information of the needle track in time to avoid the influence of the needle track abnormality on the puncture accuracy.

[0014] In a possible implementation of the first aspect, the scanning operation is performed during a posture adjustment process, or the scanning operation is performed during a needle insertion process, the posture adjustment process includes adjusting the posture of the puncture needle, and the needle insertion process includes adjusting the puncture depth of the puncture needle.

[0015] In a possible implementation of the first aspect, when the needle tip point and the predicted target point are marked on the same tomographic image, a line between the needle tip point and the predicted target point is also displayed on the marked first tomographic image. In this implementation, the line can indicate the direction of the needle track when the puncture needle is puncturing in the current puncture state, so that the user can more clearly determine the location of the needle track, and then more accurately determine whether to perform puncture according to the current puncture state.

[0016] In a possible implementation manner of the first aspect, the processor further performs the following steps: while displaying the annotated first tomographic image through the display module, also displaying adjacent tomographic images of the first tomographic image through the display module.

[0017] In a possible implementation of the first aspect, the processor further performs the following steps: the display module further displays puncture status information, the puncture status information including: at least one of puncture depth, puncture speed and posture angle, the puncture depth indicates the depth of the needle tip of the puncture needle after entering the skin, the posture angle indicates the angle of the puncture needle relative to the puncture object, and the puncture speed indicates the moving speed of the puncture needle along the needle insertion direction.

[0018] In a possible implementation of the first aspect, the processor further performs the following steps: if a first posture adjustment instruction is received, the first position information of the target lesion of the puncture object is identified from multiple tomographic images, and the first posture adjustment instruction instructs the auxiliary puncture device to automatically adjust the posture; according to the first position information and the puncture state information, the first adjustment data is obtained, and the first adjustment data represents the posture change data of the puncture needle when the predicted target point is adjusted to the first position corresponding to the first position information; the first movement instruction corresponding to the first adjustment data is sent to the surgical robot, and the first movement instruction is used to control the surgical robot to move the puncture needle for posture adjustment. In this implementation, the auxiliary puncture device obtains the position of the target lesion by performing image recognition on multiple tomographic images, wherein the position of the target lesion is the first position information, and the first adjustment data is obtained according to the first position information and the puncture state information. This method can realize automatic posture adjustment, save the user's posture adjustment time and improve the posture adjustment accuracy.

[0019] In a possible implementation of the first aspect, before sending the first movement instruction corresponding to the first adjustment data to the surgical robot, the processor is used to perform the following steps: if the processor determines that the puncture depth of the puncture needle is greater than the first threshold, or the distance between the predicted target and the first position is greater than the second threshold, or direct posture adjustment based on the first adjustment data will cause the puncture needle to interfere with the target tissue organ of the puncture object, then send a second movement instruction to the surgical robot, and the second movement instruction is used to control the surgical robot to move the puncture needle to withdraw the needle first. In this implementation, under the above three conditions, the surgical robot is controlled to withdraw the needle first and then adjust the posture, which can fully reduce the damage to the puncture object and improve the safety of automatic posture adjustment.

[0020] In a possible implementation of the first aspect, before sending a first movement instruction corresponding to the first adjustment data to the surgical robot, the first movement instruction is used to control the surgical robot to move the puncture needle for posture adjustment, the processor further performs the following steps: displaying the first adjustment data through a display module; if an instruction to use the first adjustment data is received, sending the first movement instruction corresponding to the first adjustment data to the surgical robot. In this implementation, in the automatic posture adjustment process, the user needs to confirm before the specific posture adjustment can be performed, which is equivalent to adding a step of doctor review and confirmation, thereby improving the reliability of the puncture method and the accuracy of the puncture result.

[0021] In a possible implementation of the first aspect, the processor further performs the following steps: if the second posture adjustment instruction is received, a manual posture adjustment instruction is sent to the surgical robot, the manual posture adjustment instruction being an instruction for the surgical robot to control the puncture needle to adjust the posture in response to the user's operation; the second posture adjustment instruction instructs the user to select manual posture adjustment. In this implementation, the provided manual posture adjustment process can meet the user's high autonomy requirements and facilitate the user's manual posture adjustment.

[0022] In a possible implementation of the first aspect, the scanning operation is performed by the auxiliary puncture device automatically controlling the medical imaging device. When the auxiliary puncture device automatically controls the medical imaging device to scan, the processor performs the following steps: if a prompt message for the surgical robot to perform a puncture operation is received, the medical imaging device is controlled to perform a scanning operation. The puncture operation includes a posture adjustment operation or a needle insertion operation. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle. In the implementation, the auxiliary puncture device automatically controls the medical imaging device to perform a scanning operation, thereby improving the timeliness of the scanning operation and ensuring the safety of the puncture.

[0023] In a second aspect, an embodiment of the present application provides a puncture system, the puncture system comprising: a medical imaging device, a surgical robot, and an auxiliary puncture device described in any one of the above-mentioned first aspects, wherein: the medical imaging device is used to scan the puncture object, obtain multiple tomographic images, and send the multiple tomographic images to the auxiliary puncture device; the surgical robot is used to send the puncture status information of the puncture needle in the surgical robot to the auxiliary puncture device, the puncture status information includes the status information of the puncture needle when the medical imaging device scans the puncture object; the auxiliary puncture device is used to scan the puncture object according to the puncture status information; The auxiliary puncture device is used to mark the needle tip and the predicted target point on the first tomographic image according to the position information of the needle tip and the position information of the predicted target point, and obtain the marked first tomographic image; the first tomographic image includes at least one or two tomographic images among the multiple tomographic images; the auxiliary puncture device is used to display the marked first tomographic image through the display module in the auxiliary puncture device.

[0024] In a possible implementation of the second aspect, the surgical robot includes: a surgical robot main hand, a first processor and a surgical robot slave hand, the puncture needle is arranged on the surgical robot slave hand, and the surgical robot main hand is communicatively connected with the auxiliary puncture device; wherein: the surgical robot main hand is used to send the first puncture operation instruction to the first processor after receiving the first puncture operation instruction, and send prompt information for performing the puncture operation to the auxiliary puncture device; the first processor is used to send a third movement instruction corresponding to the first puncture operation instruction to the surgical robot slave hand after receiving the first puncture operation instruction; the auxiliary puncture device is used to send a scanning instruction to the medical imaging device after receiving the prompt information; the medical imaging device is used to scan the puncture object after receiving the scanning instruction; the surgical robot slave hand is used to move the puncture needle according to the third movement instruction, and perform the puncture operation corresponding to the first puncture operation instruction, and the puncture operation includes a posture adjustment operation or a needle insertion operation, the posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

[0025] In a possible implementation of the second aspect, the surgical robot includes: a surgical robot main hand, a first processor and a surgical robot slave hand, the puncture needle is arranged on the surgical robot slave hand, and the first processor is communicatively connected with the auxiliary puncture device; wherein: the surgical robot main hand is used to send the second puncture operation instruction to the first processor after receiving the second puncture operation instruction; the first processor is used to determine a fourth movement instruction corresponding to the second puncture operation instruction; the first processor is used to send the fourth movement instruction to the surgical robot slave hand, and send prompt information for performing the puncture operation to the auxiliary puncture device; the auxiliary puncture device is used to send a scanning instruction to the medical imaging device after receiving the prompt information; the medical imaging device is used to scan the puncture object after receiving the scanning instruction; the surgical robot slave hand is used to move the puncture needle according to the fourth movement instruction, and perform the puncture operation corresponding to the second puncture operation instruction, the puncture operation includes a posture adjustment operation or a needle insertion operation, the posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

[0026] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps performed by the processor in the auxiliary puncture device as described in any one of the first aspects above.

[0027] In a fourth aspect, an embodiment of the present application provides a computer program product, which, when executed on a server, enables the server to execute the steps executed by the processor in the auxiliary puncture device described in any one of the first aspects above.

[0028] In a fifth aspect, an embodiment of the present application provides a chip, comprising: a processor, for calling and running a computer program from a memory, so that an electronic device equipped with the chip executes the steps executed by the processor in the auxiliary puncture device as described in any one of the first aspects above.

[0029] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 It is a structural schematic diagram of a puncture system provided in one embodiment of the present application;

[0032] Figure 2 This is a schematic diagram of the puncture process provided by an embodiment of the present application;

[0033] Figure 3 is a schematic diagram of displaying tissue segmentation in a preoperative image scanning stage in one embodiment of the present application;

[0034] Figure 4 is a schematic diagram of an initial puncture path displayed on a CT image provided in an embodiment of the present application;

[0035] Figure 5 is an interactive diagram of a puncture method provided in an embodiment of the present application;

[0036] Figure 6 It is a schematic diagram of an interface in which a needle tip point and a predicted target point are located in the same tomographic image in a puncture method provided in an embodiment of the present application;

[0037] Figure 7 It is a schematic diagram of an interface in which a needle tip point and a predicted target point are located in different tomographic images in a puncture method provided in an embodiment of the present application;

[0038] Figure 8 It is a schematic diagram of an interface after the needle track contour is extracted in a puncture method provided in an embodiment of the present application;

[0039] Fig. 9 It is a schematic diagram of an interface for adjusting posture on the same layer in a puncture method provided in an embodiment of the present application;

[0040] Fig.10It is a schematic diagram of an interface for cross-layer posture adjustment in a puncture method provided in an embodiment of the present application;

[0041] Fig.11 It is a structural schematic diagram of an auxiliary puncture device provided in one embodiment of the present application;

[0042] Fig.12 It is a schematic diagram of the structure of a surgical robot provided in one embodiment of the present application. DETAILED DESCRIPTION

[0043] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0044] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0045] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0046] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0047] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0048] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0049] In the current master-slave real-time guided puncture solution, doctors need to rely on clinical experience to judge the puncture path and puncture results after posture adjustment during the puncture process. Because the current master-slave real-time guided puncture solution is too dependent on the doctor's subjective medical experience, it is difficult to ensure that the puncture needle reaches the optimal puncture point after posture adjustment when the doctor is inexperienced, resulting in repeated posture adjustment, image scanning and puncture during the operation. Repeated posture adjustment, image scanning and puncture will also increase the patient's pain and radiation dose, and will prolong the operation time and increase the risk of surgery.

[0050] In order to solve the above technical problems, the present application provides an auxiliary puncture device, a puncture system and a computer-readable storage medium, wherein the auxiliary puncture device obtains multiple tomographic images obtained by a medical imaging device performing a scanning operation on a puncture object; and obtains puncture status information of a puncture needle in a surgical robot, the puncture status information includes status information of the puncture needle when the medical imaging device performs a scanning operation on the puncture object; then, based on the puncture status information, the position information of the needle tip point of the puncture needle and the position information of a predicted target point are determined, wherein the predicted target point is: the spatial point reached by the needle tip point of the puncture needle after the puncture needle completes the insertion according to the posture in the puncture status information; then, based on the position information of the needle tip point and the position information of the predicted target point, the needle tip point and the predicted target point are marked on a first tomographic image to obtain a marked first tomographic image, wherein the first tomographic image includes at least one or two tomographic images among the multiple tomographic images; and the marked first tomographic image is displayed by a display module. By marking the needle tip and the predicted target on the first tomographic image and displaying the marked first tomographic image, doctors can judge the position of the needle tip of the puncture needle and the position of the predicted target point that the needle tip can reach based on the displayed information even if they are not experienced. Therefore, the auxiliary puncture device in the embodiment of the present application can assist doctors in judging the needle tip of the puncture needle and the puncture point that the puncture needle can reach, reduce the dependence on the doctor's experience, improve the accuracy of puncture, and avoid repeated posture adjustment, image scanning and puncture during the operation.

[0051] The auxiliary puncture device, puncture system and computer-readable storage medium provided in the present application are exemplarily described below in conjunction with specific embodiments.

[0052] See also Figure 1 FIG. 1 is a schematic diagram of the structure of a puncture system provided in one embodiment of the present application. Figure 1 As shown, the puncture system in the embodiment of the present application includes: an auxiliary puncture device, a surgical robot and a medical imaging device. The auxiliary puncture device is respectively connected to the surgical robot and the medical imaging device for communication, and the auxiliary puncture device includes a display module. Among them:

[0053] The medical imaging device is used to scan the puncture object, obtain multiple tomographic images, and send the multiple tomographic images to the auxiliary puncture device.

[0054] The surgical robot is used to send the puncture status information of the puncture needle in the surgical robot to the auxiliary puncture device, and the puncture status information includes the status information of the puncture needle when the medical imaging device performs the scanning operation on the puncture object.

[0055] The auxiliary puncture device is used to determine the position information of the needle tip point of the puncture needle and the position information of the predicted target point according to the puncture status information, the predicted target point being: the spatial point reached by the needle tip point of the puncture needle after the puncture needle completes the insertion according to the posture in the puncture status information; then, according to the position information of the needle tip point and the position information of the predicted target point, the needle tip point and the predicted target point are marked on the first tomographic image to obtain the marked first tomographic image, the first tomographic image at least including one or two tomographic images among multiple tomographic images; and finally, the marked first tomographic image is displayed through the display module.

[0056] The puncture system in the embodiment of the present application, by marking the needle tip point and the predicted target on the tomographic image and displaying the marked tomographic image, enables the doctor to have a clear understanding of the position of the needle tip point of the puncture needle and the position that the needle tip point can reach (i.e., the position of the predicted target point), assists the doctor in judging the needle tip point of the puncture needle and the puncture point that the puncture needle can reach, reduces the dependence on the doctor's experience, and improves the accuracy of puncture.

[0057] like Figure 1 As shown, the surgical robot in the embodiment of the present application may include a surgical robot master hand, a first processor, and a surgical robot slave hand that are sequentially connected in communication. It can be understood that in the surgical robot, the surgical robot master hand is the operating end, and the surgical robot slave hand is the execution end. The user operates the surgical robot master hand and remotely controls the surgical robot slave hand to perform the puncture operation.

[0058] like Figure 1As shown, in order to protect doctors from the effects of radiation during medical imaging scanning, the puncture system is divided into two parts: outside the operating room and inside the operating room. The auxiliary puncture equipment operated by the doctor and the main hand of the surgical robot are set outside the operating room; inside the operating room are: medical imaging equipment for scanning the puncture object, and the slave hand of the surgical robot for puncturing the puncture object.

[0059] In some embodiments, the surgical robot slave hand may include a mechanical arm, a puncture end connected to the end of the mechanical arm, and a puncture needle installed on the puncture end. When performing a puncture operation, the mechanical arm of the surgical robot slave hand drives the puncture needle to move through the puncture end, thereby achieving a puncture operation on the puncture object. Generally speaking, the puncture operation of the puncture needle may include a posture adjustment operation and a needle insertion operation, wherein: the posture adjustment operation includes an operation of adjusting the posture of the puncture needle, and the needle insertion operation includes an operation of adjusting the puncture depth of the puncture needle.

[0060] Exemplarily, the first processor is used to convert the operation instructions applied by the doctor to the main hand of the surgical robot into movement instructions executable by the slave hand of the surgical robot, and send the movement instructions to the slave hand of the surgical robot. The slave hand of the surgical robot executes the movement instructions to drive the puncture needle to move, thereby completing the puncture operation.

[0061] It is understandable that in Figure 1 In the illustrated embodiment, the first processor is disposed in the operating room. In other embodiments, the first processor may also be disposed outside the operating room, or the first processor may also be integrated with the surgical robot's slave hand or the surgical robot's master hand. The specific location and form of the first processor in the present application are not specifically limited.

[0062] It should be understood that the medical imaging device is a medical scanning device that can obtain tomographic images, for example, it can be a CT scanning device, an MR (Magnetic Resonance) scanning device, a PET / MR (Positron Emission Tomography / Magnetic Resonance) scanning device, a PET-CT (Positron Emission Tomography-Computed Tomography) scanning device, or an ultrasonic scanning device. The present application does not limit the specific type of medical imaging device, as long as the medical imaging device can obtain multiple tomographic images that can guide the puncture process.

[0063] It is understandable that the communication connection in the embodiment of the present application may be a wireless communication connection, for example, the wireless communication connection may be implemented through wireless communication technologies such as Bluetooth (BT) technology, wireless-fidelity (WiFi) technology or near field communication (NFC) technology. Of course, the communication connection in the embodiment of the present application may also be a wired communication connection, and the present application does not limit or elaborate on this.

[0064] In some embodiments, the scanning operation of the medical imaging device on the puncture object can be manually controlled by the doctor. For example, when the doctor controls the main hand of the surgical robot to perform the puncture operation, he or she can start the medical imaging device to perform the scanning operation by stepping on the control pedal of the medical imaging device.

[0065] In some other embodiments, the scanning operation of the medical imaging device can be automatically controlled by the auxiliary puncture device.

[0066] For example, when a doctor is manipulating a surgical robot to perform a puncture operation, after receiving a first puncture operation instruction, the main hand of the surgical robot sends the first puncture operation instruction to the first processor, and sends a prompt message for the puncture operation to the auxiliary puncture device; after receiving the first puncture operation instruction, the first processor sends a third movement instruction corresponding to the first puncture operation instruction to the slave hand of the robot; after receiving the prompt message, the auxiliary puncture device sends a scanning instruction to the medical imaging device; after receiving the scanning instruction, the medical imaging device performs a scanning operation on the puncture object; the slave hand of the surgical robot moves the puncture needle according to the third movement instruction, and performs a puncture operation corresponding to the first puncture operation instruction. The puncture operation includes a posture adjustment operation or a needle insertion operation. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle. It can be understood that the first puncture operation instruction can be the doctor's operation on the main hand of the surgical robot, the prompt message can be a notification sent by the main hand of the surgical robot to the auxiliary puncture device surgical robot that a puncture operation is about to be performed, and the puncture object can be a patient. In this embodiment, after receiving the first puncture operation instruction, the main hand of the surgical robot prompts the auxiliary puncture device to control the medical imaging device to perform a scanning operation, so that the medical imaging device can be automatically controlled to perform real-time exposure scanning during the puncture operation, thereby ensuring that the medical imaging device will perform a scanning operation every time the main hand of the surgical robot receives a puncture operation instruction, thereby ensuring the timeliness of the medical imaging scan. In this embodiment, the medical imaging scan is performed using the time interval between the main hand of the surgical robot receiving the puncture operation instruction and the slave hand of the surgical robot executing the puncture operation instruction, so that the medical imaging scan can be performed before the puncture operation is performed, thereby avoiding blind puncture caused by the puncture operation having been performed but the medical imaging scan not being performed.

[0067] It can be understood that the specific function of the third movement instruction is related to the puncture operation corresponding to the first puncture operation instruction. When the puncture operation corresponding to the first puncture operation instruction is a posture adjustment operation, the third movement instruction is used to control the surgical robot to move the puncture needle for posture adjustment; when the puncture operation corresponding to the first puncture operation instruction is a needle insertion operation, the third movement instruction is used to control the surgical robot to move the puncture needle for needle insertion.

[0068] For another example, when the doctor is operating the surgical robot to perform a puncture operation, the main hand of the surgical robot sends the second puncture operation instruction to the first processor after receiving the second puncture operation instruction. The first processor determines the fourth movement instruction corresponding to the second puncture instruction, and then sends the fourth movement instruction to the slave hand of the surgical robot, and sends a prompt message for the puncture operation to the auxiliary puncture device. After receiving the prompt message, the auxiliary puncture device sends a scanning instruction to the medical imaging device. After receiving the scanning instruction, the medical imaging device performs a scanning operation on the puncture object; the slave hand of the surgical robot moves the puncture needle according to the fourth movement instruction to perform the puncture operation corresponding to the second puncture operation instruction. The puncture operation includes a posture adjustment operation or a needle insertion operation. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle. It can be understood that the second puncture operation instruction can be the doctor's operation on the main hand of the surgical robot, the prompt message can be the notification from the first processor to the auxiliary puncture device that the surgical robot is about to perform a puncture operation, and the puncture object can be a patient. In this embodiment, after receiving the second puncture operation instruction, the main hand of the surgical robot sends the second puncture instruction to the first processor. After receiving the second puncture operation instruction, the first processor prompts the auxiliary puncture device to control the medical imaging device to perform a scanning operation, so that the medical imaging device can be automatically controlled to perform real-time exposure during the puncture operation, thereby ensuring that the medical imaging device will perform a scanning operation every time the main hand of the surgical robot receives a puncture operation instruction, thereby ensuring the timeliness of the medical imaging scan; in this embodiment, the first processor sends a fourth movement instruction corresponding to the second puncture operation instruction to the slave hand of the surgical robot, and the time interval between the execution of the fourth movement instruction by the slave hand of the surgical robot is used to perform a medical imaging scan, so that the medical imaging scan can be performed before the puncture operation is performed, thereby avoiding blind puncture caused by the puncture operation having been performed but no medical imaging scan. In addition, the first processor sends the instruction information to the auxiliary puncture device, and the scanning time will be shortened to a certain extent compared with the main hand of the surgical robot sending the instruction information to the auxiliary puncture device, so the radiation amount of the puncture object can be reduced.

[0069] It can be understood that the specific function of the fourth movement instruction is related to the puncture operation corresponding to the second puncture operation instruction. When the puncture operation corresponding to the second puncture operation instruction is a posture adjustment operation, the fourth movement instruction is used to control the surgical robot to move the puncture needle for posture adjustment; when the puncture operation corresponding to the second puncture operation instruction is a needle insertion operation, the fourth movement instruction is used to control the surgical robot to move the puncture needle for needle insertion.

[0070] It should be understood that the first puncture operation instruction and the second puncture operation instruction may be the same or different, and the present application does not impose any limitation on this.

[0071] After introducing the puncture system, Figure 1 The puncture system shown is an exemplary description of the puncture process based on the puncture system in the embodiment of the present application.

[0072] Figure 2 This is a schematic diagram of the puncture process provided by an embodiment of the present application. Figure 2 As shown, the puncture process mainly includes: S201 to S206, and each step is described below.

[0073] S201, preoperative imaging scan.

[0074] Specifically, before the puncture operation begins, the diseased part of the puncture object is scanned using medical imaging equipment. For example, when the medical imaging equipment is a CT scanning device, the CT scanning device can be used to perform multi-phase enhanced scanning and plain scanning. The images obtained by scanning can clarify the nature, outline and range of the lesion, and can realize the analysis of important organs.

[0075] For example, during the preoperative scanning stage, the abdominal liver and the thoracic lung lobes can be reconstructed in three dimensions in the transverse coronal and sagittal directions, and displayed using VR (Virtual Reality).

[0076] exist Figure 2 In the illustrated embodiment, the tumor site is used as the target location for puncture, and operations such as target organ segmentation, tumor site extraction, and dangerous tissue identification can be achieved during the preoperative image scanning stage.

[0077] For example, during the preoperative image scanning stage, the contours of important tissues and organs such as lesions, soft tissues, and target organs can be extracted based on the Dicom images obtained by scanning with a CT scanning device, and the extracted images can be displayed through a display module.

[0078] For example, Figure 3 FIG. 1 is a schematic diagram of tissue segmentation obtained during the preoperative image scanning stage in an embodiment of the present application. Figure 3As shown, in this embodiment, a CT scanning device is used to perform a scanning operation, wherein the lesion is located on the liver. Figure 3 The outline of the lesion 301 is marked in FIG. In this embodiment, in the preoperative image scanning stage, marking the lesion in the image can assist the doctor in locating the lesion position of the patient and understanding the situation around the lesion.

[0079] S202, initial puncture path planning.

[0080] Based on the tissue model obtained during the preoperative imaging scan, Figure 3 The tissue segmentation diagram shown automatically plans the initial puncture path while avoiding important tissues and organs such as blood vessels, nerves, and bones.

[0081] The specific contents of the initial puncture path may include: the puncture point on the body surface of the puncture object, the puncture target point, the needle insertion angle, and the needle insertion depth. Figure 4 This is a schematic diagram of an initial puncture path displayed on an image obtained by a CT scanning device provided in an embodiment of the present application. Figure 4 As shown, in this embodiment, the initial puncture path includes a body surface puncture point 401, a puncture target point 402, and a needle track 403 connected between the body surface puncture point 401 and the puncture target point 402, wherein the puncture target point 402 is located within the lesion contour 404.

[0082] In some embodiments, the initial puncture path may also be displayed on a 3D image, wherein the conversion of a two-dimensional image to a 3D image may be achieved by three-dimensional reconstruction, which is not elaborated in this application.

[0083] S203, intraoperative navigation and positioning.

[0084] The end puncture execution device is connected to the hand of the surgical robot, and the end puncture device is fixed next to the medical imaging device. Then, the surgical robot navigation device determines the registration relationship between the puncture object space, which can also be called the patient space, the mechanical arm space of the surgical robot hand, and the medical imaging space of the medical imaging device, and obtains the registration matrix. The coordinate conversion between the three spaces can be realized by using the registration rectangle.

[0085] S204, intraoperative puncture preparation.

[0086] The doctor selects a puncture needle of appropriate length according to the insertion depth in the initial puncture path planning, and installs the puncture needle on the puncture end of the surgical robot's slave hand. After the puncture needle is installed, the surgical robot's slave hand automatically controls the mechanical arm to drive the puncture needle to the puncture starting point, where the puncture starting point is the surface puncture point in the initial path planning, and adjusts the puncture needle's insertion angle to be consistent with the insertion angle in the initial puncture path planning.

[0087] S205, intraoperative puncture.

[0088] like Figure 2 As shown in the figure, the intraoperative puncture stage mainly includes three parts: needle track posture adjustment, needle track insertion, and real-time exposure. Needle track posture adjustment mainly includes the adjustment of the posture of the puncture needle, and needle track insertion mainly includes the adjustment of the puncture depth of the puncture needle; real-time exposure mainly includes scanning the medical image of the puncture object during the posture adjustment and needle insertion process.

[0089] In the embodiment of the present application, the doctor can choose automatic posture adjustment or manual posture adjustment according to the clinical situation. Automatic posture adjustment means that the auxiliary puncture device automatically controls the surgical robot to adjust the posture of the puncture needle. The manual posture adjustment process means that the doctor controls the surgical robot to adjust the posture of the puncture needle in the surgical robot by translation, rotation, etc.

[0090] In an embodiment of the present application, during the process of needle track posture adjustment and needle insertion, real-time exposure is performed by controlling the medical imaging device, and the exposure range is set within the puncture needle insertion range, that is, the exposure range is set within the slice range where the needle track is located, so as to observe in real time the movement direction of the puncture needle in the patient's body and the actual position reached by the needle tip of the puncture needle, and then judge the current puncture situation, thereby avoiding the impact of the puncture on the patient's important tissues and organs, where the patient is the puncture object.

[0091] S206, completing the puncture.

[0092] After the needle is inserted into the needle tract, the needle tip of the puncture needle reaches the lesion area, completing the puncture operation.

[0093] It is understandable that the lesion area in the embodiments of the present application is a broad concept, and the definition of the lesion area may vary depending on the type of surgery. For example, for puncture surgeries such as biopsy and injection, the lesion area refers to the area surrounded by the lesion outline; for puncture surgeries such as ablation and particle implantation, the lesion area includes the area surrounded by the lesion outline, as well as the area within a certain range of the lesion edge or periphery.

[0094] It is understandable that in the above puncture process, the intraoperative puncture stage is the most critical and important stage. The puncture method in the embodiment of the present application is mainly improved for the intraoperative puncture stage. The puncture method in the embodiment of the present application is described below in conjunction with the accompanying drawings.

[0095] See also Figure 5 , is an interactive diagram of an example of a puncture method provided in this application. This method is applicable to Figure 1 The puncture system shown below is combined with Figure 5 The puncture method in the embodiment of the present application is described. Figure 5 As shown, the puncture method includes: S501 to S506.

[0096] S501, a medical imaging device performs a scanning operation on a puncture object to obtain a plurality of tomographic images.

[0097] S502, the medical imaging device sends a plurality of tomographic images to the auxiliary puncture device.

[0098] In some embodiments, the scanning operation may be performed in Figure 2 The scanning operation performed before the intraoperative puncture shown in . For example, it can be the scanning operation performed in the preoperative imaging scanning stage.

[0099] In some other embodiments, the scanning operation can also be performed during the posture adjustment process or during the needle insertion process, wherein the posture adjustment process includes adjusting the posture of the puncture needle, and the needle insertion process includes adjusting the puncture depth of the puncture needle. Performing the scanning operation during the posture adjustment or needle insertion process can ensure that the scanning operation is performed during the main stage of puncture, ensuring that the user can monitor the puncture process in real time through the tomographic image, reducing the difficulty of puncture and improving the accuracy of puncture.

[0100] In some embodiments, during the posture adjustment process, the scanning device performs a scanning operation before the surgical robot performs the posture adjustment operation; during the needle insertion process, the scanning device performs a scanning operation before the surgical robot performs the needle insertion operation. By performing scanning before the surgical robot performs the posture adjustment operation or the needle insertion operation, the guiding effect of the scanning on the posture adjustment operation or the needle insertion operation can be ensured to avoid blind posture adjustment or needle insertion.

[0101] S503, the surgical robot sends the puncture status information of the puncture needle in the surgical robot to the auxiliary puncture device, where the puncture status information includes the status information of the puncture needle when the medical imaging device performs a scanning operation on the puncture object.

[0102] Exemplarily, the puncture state information of the puncture needle may include: at least one of the puncture depth, puncture speed and posture angle, wherein the puncture depth indicates the depth of the needle tip of the puncture needle into the skin, the posture angle indicates the angle of the puncture needle relative to the puncture object, and the puncture speed indicates the moving speed of the puncture needle along the needle insertion direction. Alternatively, the puncture needle puncture speed can also be understood as the moving speed of the puncture needle along the direction where its length is located.

[0103] In some embodiments, the auxiliary puncture device displays puncture status information through a display module. Exemplarily, the auxiliary puncture device displays at least one of puncture depth, puncture speed, and posture angle through a display module. By displaying the puncture status information, the user can see the puncture status information of the puncture needle at any time, thereby improving the puncture accuracy.

[0104] It should be understood that the puncture status information may be theoretical data when the surgical robot performs a puncture operation. In this case, the puncture status information may be obtained through theoretical calculations. For example, the puncture status information may be determined based on the theoretical data on which the surgical robot performs the puncture operation. Details will not be given here.

[0105] In addition, the puncture status information may also be the actual status information of the surgical robot when performing the puncture operation. In this case, the puncture status information can be obtained by real-time tracking and identification of the actual position of the puncture needle. For example, the actual position of the puncture needle can be tracked and identified by performing image analysis on the tomographic images obtained during the puncture process, and the actual position of the puncture needle can be obtained. The puncture speed is determined according to the actual position information of the puncture needle; the actual position of the puncture needle and the position information of the puncture object are used to determine the puncture depth and posture angle, which will not be elaborated here.

[0106] S504, the auxiliary puncture device determines the position information of the needle tip of the puncture needle and the position information of the predicted target point according to the puncture state information, and the predicted target point is: the spatial point reached by the needle tip of the puncture needle after the puncture needle completes the insertion according to the posture in the puncture state information.

[0107] In some embodiments, the puncture state information may also include the position information of the needle tip point, which is calculated based on the relative position relationship between the puncture needle and the surgical robot, that is, the position information of the needle tip point is a calculated theoretical value. When the puncture needle is in a normal state, the calculated position information of the needle tip point is accurate. In this case, the auxiliary puncture device directly obtains the position information of the needle tip point from the puncture state information. It can be understood that the puncture needle being in a normal state may mean that the puncture needle does not have abnormal conditions such as bending or breaking.

[0108] Exemplarily, the position information of the needle tip point may include the coordinates of the needle tip point, which can be obtained by solving the joint angles of the robot arm. The specific calculation process will not be described in detail here.

[0109] In some other embodiments, in addition to directly obtaining the coordinates of the needle tip point from the surgical robot, the auxiliary puncture device can also obtain relevant parameters from the surgical robot and calculate the joint angles of the robotic arm according to the relevant parameters to obtain the coordinates of the needle tip point.

[0110] In the embodiment of the present application, the predicted target point is a prediction of a spatial point that the puncture needle may reach. Exemplarily, the predicted target point position includes the predicted target point coordinates. Specifically, the predicted target point coordinates can be obtained based on the initial coordinates of the needle tip in the robot coordinate system and the puncture needle length. The specific calculation process is not described in detail here.

[0111] S505, the auxiliary puncture device marks the needle tip point and the predicted target point on the first tomographic image according to the position information of the needle tip point and the position information of the predicted target point, and obtains the marked first tomographic image, and the first tomographic image includes at least one or two tomographic images among the multiple tomographic images.

[0112] Exemplarily, the needle tip point is marked on the first tomographic image. Specifically, after determining the tomographic image where the needle tip point is located, the coordinates of the needle tip point are converted from the robot coordinate system to the image coordinate system where multiple tomographic images obtained by the medical imaging device are located, and then the needle tip point is marked on the corresponding tomographic image. The specific coordinate conversion process is not elaborated here.

[0113] It should be understood that the needle tip and the predicted target point occupy a certain three-dimensional space respectively. The needle tip can be located at one or more fault layers of the puncture object, so the fault images related to the needle tip can include one or more; similarly, the predicted target point can also be located at one or more fault layers of the puncture object, so the fault images related to the predicted target point can include one or more.

[0114] For example, when the needle tip point and the predicted target point are located in the same tomographic layer of the puncture object, the needle tip point and the predicted target point can be marked on the same tomographic image, in which case the first tomographic image includes one tomographic image among multiple tomographic images; when the needle tip point and the predicted target point are located in different tomographic layers of the puncture object, the needle tip point and the predicted target point are marked on two different tomographic images, in which case the first tomographic image includes at least two tomographic images among multiple tomographic images.

[0115] S506: The auxiliary puncture device displays the annotated first tomographic image through a display module in the auxiliary puncture device.

[0116] It is understandable that the annotated first tomographic image displayed in the display module may be a partial tomographic image in the first tomographic image, or may be all tomographic images in the first tomographic image, and the present application does not impose any limitation on this.

[0117] For example, the display module can display the tomographic image where the center of the needle tip point is located among all tomographic images marked with the needle tip point; and the display module can display the tomographic image where the center of the predicted target point is located among all tomographic images marked with the predicted target point.

[0118] In some embodiments, when the needle tip point and the predicted target point are located in the same layer of the puncture object, the needle tip point and the predicted target point will be marked on the same tomographic image. At this time, the tomographic image displayed by the display module can be a tomographic image marked with the needle tip point and the predicted target point.

[0119] For example, Figure 6 The figure shows a tomographic image displayed by a display module in an auxiliary puncture device in an example of the present application. Figure 6 As shown, in this embodiment, the medical imaging device is a CT scanning device. In this embodiment, a total of 4 different tomographic images are displayed, wherein the needle tip point 601 and the predicted target point 602 are located on the same tomographic image.

[0120] For example, when the needle tip point and the predicted target point are marked on the same tomographic image, a line connecting the needle tip point and the predicted target point is also displayed on the first tomographic image after marking. Figure 6 In the figure, a line 603 is also displayed between the needle tip point 601 and the predicted target point 602. The line 603 can indicate the direction of the needle track when the puncture needle is punctured in the current puncture state, so that the user can more clearly determine the position of the needle track, and then more accurately determine whether to perform puncture according to the current puncture state.

[0121] It is understandable that if Figure 6 The figure shows a tomographic image of the needle insertion state, so the needle tip point 601 and the predicted target point 602 are located on the same tomographic image, so the needle insertion process is the same layer needle insertion. At this time, the needle tip point and the predicted target point are both on the -729.125mm layer. At this time, the two marked points are connected by a line 603, where the line 603 can be a dotted line to indicate the subsequent needle insertion path of the needle tip, which is convenient for doctors to judge the tissue information that will be passed during the subsequent puncture process. As the needle insertion depth increases, the needle tip point 601 will continue to approach the predicted target point 602 along the dotted line until the puncture is completed.

[0122] In some other embodiments, when the needle tip point and the predicted target point are located in two different slices of the puncture object, the needle tip point and the predicted target point will be marked on two different slice images.

[0123] For example, Figure 7 The figure shows a tomographic image displayed by a display module in an auxiliary puncture device in an example of the present application. Figure 7 As shown, in this embodiment, the medical imaging device is a CT scanning device. In this embodiment, a total of 4 different tomographic images are displayed, wherein the needle tip point 701 and the predicted target point 702 are located on two different tomographic images.

[0124] It is understandable that if Figure 7 The tomographic image of the needle insertion state is shown, so the needle tip point 701 and the predicted target point 702 are located on different tomographic images, so the needle insertion process is cross-layer needle insertion. At this time, the real-time needle tip point 701 is at the -729.125mm layer, and the predicted target point 702 is at the -727.875mm layer. As the needle insertion depth increases, the real-time needle tip point 701 will puncture from the initial layer (i.e., the -729.125mm layer) to the target layer (i.e., the -727.875mm layer) until the puncture is completed.

[0125] It is understandable that in Figure 6 and Figure 7 In order to facilitate the user to distinguish, marks with different colors and shapes can be used to mark the needle tip point and the predicted target point. Of course, in other embodiments, marks with the same shape but different colors can be used to mark the needle tip point and the predicted target point, or marks with the same color but different shapes can be used to mark the needle tip point and the predicted target point. This application does not limit the specific form of the mark.

[0126] In some embodiments, while displaying the annotated first tomographic image through the display module, the adjacent tomographic images of the first tomographic image are also displayed through the display module. By displaying the adjacent tomographic images, the movement of the needle tip and the predicted target point in different tomographic layers can be seen during puncture; at the same time, when the needle tip and the predicted target point move to the adjacent tomographic image, they can be quickly displayed.

[0127] It is understandable that the adjacent tomographic image of the first tomographic image may be the tomographic image corresponding to the tomographic layer corresponding to the first tomographic image and the tomographic image corresponding to the previous tomographic layer; or may be the tomographic image corresponding to the tomographic layer corresponding to the first tomographic image and the image corresponding to the subsequent tomographic layer. Furthermore, the tomographic image corresponding to the previous tomographic layer may include one or more tomographic images, and the tomographic image corresponding to the subsequent tomographic layer may also include one or more tomographic images, which is not limited in the present application.

[0128] For example, in Figure 6 In the embodiment shown, in addition to the first tomographic image marked with the needle tip point 601 and the predicted target point 602, Figure 6 Also shown in FIG. 3 are three tomographic images adjacent to the first tomographic image.

[0129] For example, in Figure 7 In the embodiment shown, in addition to the two different first tomographic images marked with the needle tip point 701 and the predicted target point 702, Figure 7 Also shown in FIG. 1 are two tomographic images adjacent to the first tomographic image, wherein each of the two tomographic images is adjacent to one of the first tomographic images.

[0130] In some embodiments, the auxiliary puncture device displays puncture status information through a display module. The displayed puncture status information may include: at least one of puncture depth, puncture speed and posture angle, wherein the puncture depth indicates the depth of the needle tip of the puncture needle after entering the skin, the posture angle indicates the angle of the puncture needle relative to the puncture object, and the puncture speed indicates the moving speed of the puncture needle along the needle insertion direction.

[0131] In some embodiments, when the auxiliary puncture device determines that the puncture speed is greater than a certain threshold, the display module displays a warning message that the puncture is too fast. By reminding that the puncture speed is too fast, the probability of too fast puncture can be reduced and the safety of the puncture operation can be improved. For example, when the puncture speed is greater than 30mm / s, a warning message that the puncture is too fast is displayed. The specific form of the warning message can be a pop-up prompt window, or the display mode of the puncture speed changes, such as displaying a warning icon while displaying the puncture speed. This application does not limit the form of the warning message.

[0132] In some other embodiments, the auxiliary puncture device can also: extract the needle track contour of the puncture needle based on multiple tomographic images, the needle track contour represents the three-dimensional structure of the puncture needle; determine the actual position information of the needle tip of the puncture needle based on the needle track contour, the actual position information of the needle tip represents the position information of the needle tip of the puncture needle in the needle track contour; if it is determined that the needle track contour is abnormal based on the position information of the needle tip and the actual position information of the needle tip, then generate a needle track abnormality warning. In this embodiment, the needle track abnormality warning can prompt the user with the abnormal information of the needle track in time to avoid the influence of the needle track abnormality on the puncture accuracy.

[0133] For ease of understanding, the following is an exemplary description of the process of determining whether there is an abnormality in the needle track profile based on the position information of the needle tip point and the actual position information of the needle tip point.

[0134] Exemplarily, the position information of the needle tip point includes the theoretical coordinates of the needle tip point, which are the coordinates of the needle tip point reported by the surgical robot; the actual position information of the needle tip point includes the actual coordinates of the needle tip point, which are obtained based on the extracted needle track contour; since the actual coordinates of the needle tip point are located in the image coordinate system, when determining whether there is an abnormality in the needle track contour, the actual coordinates of the needle tip point are first converted to the robot coordinate system to obtain the converted actual coordinates of the needle tip point; then, in the robot coordinate system, the converted actual coordinates of the needle tip point are compared with the theoretical coordinates of the needle tip point; if the distance between the converted actual coordinates of the needle tip point and the theoretical coordinates of the needle tip point is greater than a preset threshold, it is determined that there is an abnormality in the needle track contour; if the distance between the converted actual coordinates of the needle tip point and the theoretical coordinates of the needle tip point is less than or equal to the preset threshold, it is determined that there is no abnormality in the needle track contour.

[0135] In some embodiments, when determining whether there is an abnormality in the needle path contour, the theoretical coordinates of the needle tip point in the robot coordinate system can be first converted to the image coordinate system to obtain the converted theoretical coordinates of the needle tip point; then, in the image coordinate system, the converted theoretical coordinates of the needle tip point are compared with the actual coordinates of the needle tip point. If the distance between the converted theoretical coordinates of the needle tip point and the actual coordinates of the needle tip point is greater than a preset threshold, it is determined that there is an abnormality in the needle path contour; if the distance between the converted theoretical coordinates of the needle tip point and the actual coordinates of the needle tip point is less than or equal to the preset threshold, it is determined that there is no abnormality in the needle path contour.

[0136] For example, Figure 8 is a schematic diagram of the needle track contour extracted in an embodiment of the present application, such as Figure 8 As shown, the medical imaging device in this embodiment is a CT scanning device. Figure 8 Figure a is a tomographic image before needle track contour extraction. Figure 8 Figure b in the figure is a tomographic image after the needle track contour is extracted. Figure 8 In Figure b, the target lesion is also marked, namely Figure 8 FIG. 8 b shows a needle track outline 801 and a target lesion 802 .

[0137] It can be understood that the needle track contour of the puncture needle can be obtained by three-dimensionally reconstructing the puncture needle based on multiple tomographic images, or it can be obtained by using an image recognition algorithm, where the image recognition algorithm can be a machine learning algorithm; both the three-dimensional reconstruction and the image recognition algorithm can use common algorithms, and this application does not limit or elaborate on this.

[0138] Exemplarily, the needle tract abnormality warning generated by the auxiliary puncture device can be an abnormality warning displayed by a display module, for example, needle tract abnormality prompt information can be displayed on the display module, and the needle tract abnormality prompt information can be a prompt window; it can also be an abnormality warning issued in other forms, for example, an alarm module can be installed in the auxiliary puncture device, and an abnormal alarm can be issued by the alarm module. This application does not limit the specific form of the needle tract abnormality warning.

[0139] In some other embodiments, the auxiliary puncture device can also mark the target lesion of the puncture object. Specifically, the auxiliary puncture device marks the target lesion of the puncture object on the second tomographic image among the multiple tomographic images to obtain the marked second tomographic image, which includes the tomographic image of the target lesion scanned among the multiple tomographic images; the marked second tomographic image is displayed by the display module. In this embodiment, by marking the target lesion, the user can see the lesion of the puncture object more clearly, so that the user can control the puncture process more accurately and improve the user experience.

[0140] It is understandable that the lesion is generally a three-dimensional structure, so the target lesion of the puncture object may exist in multiple slices, that is, the target lesion may be scanned by multiple slice images.

[0141] In some embodiments, the second tomographic image includes only one tomographic image, then the second tomographic image may be a tomographic image that scans the center of the target lesion, or the second tomographic image may be a tomographic image with the largest lesion area, or the second tomographic image may be a tomographic image that meets other conditions, and the present application does not impose any restrictions on this.

[0142] In some other embodiments, the second tomographic image may also include multiple tomographic images, or the second tomographic image may also include all tomographic images of the target lesion scanned, which is not limited in the present application.

[0143] In some embodiments, the target lesion is also marked on the tomographic image marked with the predicted target, that is, the first tomographic image marked with the predicted target and the second tomographic image marked with the target lesion can be the same tomographic image. By marking the target lesion on the tomographic image where the predicted target is located, it is convenient for the user to judge whether the target lesion can be punctured by puncturing with the current puncture state of the puncture needle. For example, when the predicted target coincides with the target lesion, or the predicted target is within the contour range of the target lesion, it can be determined that the target lesion can be punctured by puncturing with the current puncture state of the puncture needle. Otherwise, it is determined that the target lesion cannot be punctured by puncturing with the current puncture state of the puncture needle.

[0144] In some other embodiments, the needle tip point, predicted target point and target lesion can be marked on the same tomographic image, the first tomographic image marked with the needle tip point and the predicted target point includes a tomographic image, the second tomographic image marked with the target lesion includes a tomographic image, and the first tomographic image and the second tomographic image are the same tomographic image.

[0145] Exemplarily, the auxiliary puncture device displays the annotated second tomographic image through the display module, and also displays the adjacent tomographic images of the second tomographic image through the display module. The display of the adjacent tomographic images of the second tomographic image in this embodiment is basically the same as the display of the adjacent tomographic images of the first tomographic image in terms of method and technical effect, and will not be described in detail here.

[0146] In some embodiments, the auxiliary puncture device also displays the three-dimensional image of the puncture needle and the three-dimensional image of the target lesion through the display module, and the three-dimensional image of the puncture needle and the three-dimensional image of the target lesion are obtained using multiple tomographic images respectively. Among them, the three-dimensional image of the puncture needle and the three-dimensional image of the target lesion can be obtained by three-dimensional reconstruction using multiple tomographic images respectively, and of course, they can also be generated by an image recognition algorithm, which is not limited in this application. In this embodiment, by displaying the three-dimensional image, the presentation of the puncture needle and the target lesion is made more three-dimensional and vivid.

[0147] Exemplarily, the puncture needle and the target lesion are displayed in the same three-dimensional image, and the relative position relationship between the puncture needle and the target lesion in the three-dimensional image is the same as the relative position relationship between the puncture needle and the target lesion in the image coordinate system, where the image coordinate system is the coordinate system where the multiple tomographic images are located. By displaying the three-dimensional image of the puncture needle and the target lesion in the same three-dimensional image, and faithfully presenting the relative position relationship between the puncture needle and the target lesion in the image coordinate system, the user can more intuitively see the puncture progress, thereby facilitating the user to judge whether the puncture is completed.

[0148] For example, in the same three-dimensional image, the puncture needle and the target lesion are displayed in different colors. By displaying the puncture needle and the target lesion in different colors, it is convenient for the user to distinguish the puncture needle and the target lesion, and the user can better judge the relative position relationship between the puncture needle and the target lesion.

[0149] It should be understood that when the user determines that the predicted target point does not overlap with the target lesion based on the image displayed by the display module, or that the predicted target point is outside the contour of the target lesion, it is necessary to perform a posture adjustment operation of the puncture needle. For ease of understanding, the posture adjustment process in the puncture method in the embodiment of the present application is exemplarily described below in conjunction with the accompanying drawings.

[0150] In some embodiments, the puncture system provides two posture adjustment methods: automatic posture adjustment and manual posture adjustment, and the user can choose according to the clinical situation. The automatic posture adjustment and manual posture adjustment are respectively described exemplarily below.

[0151] When the user selects automatic posture adjustment, the auxiliary puncture device provides adjustment data for posture adjustment.

[0152] In some embodiments, the auxiliary puncture device can also: identify the first position information of the target lesion of the puncture object from multiple tomographic images when receiving the first posture adjustment instruction; obtain the first adjustment data according to the first position information and the puncture state information, the first adjustment data representing the posture change data of the puncture needle when the predicted target point is adjusted to the first position corresponding to the first position information; send the first movement instruction corresponding to the first adjustment data to the surgical robot, the first movement instruction is used to control the surgical robot to move the puncture needle for posture adjustment; wherein the first posture adjustment instruction instructs the auxiliary puncture device to automatically adjust the posture, that is, the user selects the automatic posture adjustment function. In this embodiment, the auxiliary puncture device obtains the position of the target lesion by performing image recognition on multiple tomographic images, wherein the position of the target lesion is the first position information, and obtains the first adjustment data according to the first position information and the puncture state information. This method can realize automatic posture adjustment and save the user's posture adjustment time.

[0153] In some embodiments, before the auxiliary puncture device sends the first movement instruction corresponding to the first adjustment data to the surgical robot, the first adjustment data is displayed in the display module, and if an instruction to use the first adjustment data is received, the first movement instruction corresponding to the first adjustment data is sent to the surgical robot. That is, in the automatic posture adjustment process in this embodiment, the user needs to confirm before the specific posture adjustment can be performed, which is equivalent to adding a step of doctor review and confirmation, thereby improving the reliability of the puncture method and the accuracy of the puncture result.

[0154] In some embodiments, when performing automatic posture adjustment, the auxiliary puncture device can also determine whether it is necessary to withdraw the needle first and then adjust the posture. Exemplarily, before sending the first movement instruction corresponding to the first adjustment data to the surgical robot, the auxiliary puncture device is used to perform the following steps: if the auxiliary puncture device determines that the puncture depth of the puncture needle is greater than the first threshold, or the predicted distance between the target point and the first position is greater than the second threshold, or direct posture adjustment based on the first adjustment data will cause the puncture needle to interfere with the target tissue organ of the puncture object, the auxiliary puncture device sends the second movement instruction to the surgical robot, and the second movement instruction is used to control the surgical robot to move the puncture needle to withdraw the needle; after completing the needle withdrawal, the processor sends the first movement instruction corresponding to the first adjustment data to the surgical robot, and the first movement instruction is used to control the surgical robot to move the puncture needle for posture adjustment.

[0155] It is understandable that, in the case of automatic posture adjustment, if the puncture needle's puncture depth is greater than the first threshold, it indicates that the puncture needle has punctured to a deeper depth, and direct posture adjustment may cause greater damage to the puncture object; if the distance between the predicted target and the first position is greater than the second threshold, it indicates that the predicted target deviation angle is large. In this case, if the puncture needle needs to be adjusted by a large amplitude when directly adjusting the posture, it will cause greater damage to the puncture object; the puncture needle and the target tissue organ of the puncture object interfere with each other, which means that the puncture needle may touch the target tissue organ of the puncture object, and then may injure the touched target tissue organ. Therefore, under these three conditions, controlling the surgical robot to withdraw the needle first and then adjust the posture can fully reduce the damage to the puncture object and improve the safety of automatic posture adjustment. Among them, the target tissue organ refers to the important tissue organ of the puncture object, for example, the target tissue organ can be an important tissue organ such as blood vessels, bones, nerves, etc.

[0156] In some other embodiments, when performing automatic posture adjustment, before sending the first movement instruction corresponding to the first adjustment data to the surgical robot, the auxiliary puncture device is used to perform the following steps: if the auxiliary puncture device determines that the puncture depth of the puncture needle is less than or equal to the first threshold, and predicts that the distance between the target point and the first position is less than or equal to the second threshold, and direct posture adjustment based on the first adjustment data will not cause interference between the puncture needle and the target tissue organ of the puncture object, the auxiliary puncture device sends the first movement instruction to the surgical robot, and the first movement instruction is used to control the surgical robot to move the puncture needle to adjust the posture.

[0157] It can be understood that in the case of automatic posture adjustment, if the puncture depth of the puncture needle is less than or equal to the first threshold, it means that the puncture depth of the puncture needle is shallow, and direct posture adjustment will cause less damage to the puncture object; if the distance between the predicted target and the first position is less than or equal to the second threshold, it means that the predicted target deviation angle is small, in which case the direct posture adjustment of the puncture needle requires a smaller adjustment range, and the damage to the puncture object is also smaller; the puncture needle does not interfere with the target tissue and organ of the puncture object, which means that the puncture needle will not touch the target tissue and organ of the puncture object. Therefore, when all three conditions are met, the surgical robot can be controlled to adjust the posture directly without withdrawing the needle.

[0158] Exemplarily, after the auxiliary puncture device determines that the posture can be adjusted directly without withdrawing the needle, before sending the first movement instruction corresponding to the first adjustment data to the surgical robot, the first adjustment data is displayed in the display module, and if an instruction to use the first adjustment data is received, the first movement instruction corresponding to the first adjustment data is sent to the surgical robot. That is, in the automatic posture adjustment process in this embodiment, the user needs to confirm before the specific posture adjustment can be performed, which is equivalent to adding a step of doctor review and confirmation, thereby improving the reliability of the puncture method and the accuracy of the puncture results.

[0159] In some embodiments, when performing manual posture adjustment, the user controls the surgical robot to drive the puncture needle to adjust the posture. Exemplarily, the auxiliary puncture device can also: when receiving the second posture adjustment instruction, the auxiliary puncture device sends a manual posture adjustment instruction to the surgical robot, and the manual posture adjustment instruction is an instruction for the surgical robot to control the puncture needle to adjust the posture in response to the user's operation, that is, in this embodiment, the user selects manual posture adjustment. In this embodiment, the manual posture adjustment process provided can meet the user's high autonomy requirements and facilitate the user's manual posture adjustment.

[0160] It is understandable that in the specific posture adjustment process, the posture adjustment may include: same-layer posture adjustment, cross-layer posture adjustment, and free mode posture adjustment. The same-layer posture adjustment, cross-layer posture adjustment, and free mode posture adjustment are exemplarily described below in conjunction with the accompanying drawings. It is understandable that the medical imaging device in the following example is a CT scanning device.

[0161] (1) Adjust posture on the same level.

[0162] Same-layer posture adjustment means that the predicted target before and after posture adjustment are in the same transverse position (i.e., on the same tomographic image). In this case, only the same-layer angle (i.e., left-right deflection angle) needs to be changed, and the cross-layer angle (i.e., head-foot deflection angle) does not need to be changed. Fig. 9 As shown, in Fig. 9 Figure a is the tomographic image before posture adjustment. Fig. 9 The b figure in the figure is the tomographic image after posture adjustment. Fig. 9 As shown in Figure a, the predicted target 901 and the target lesion 902 are located on the same tomographic image before posture adjustment, but they do not overlap, so same-layer posture adjustment is required. After the same-layer angle is changed, the position of the predicted target changes accordingly on the Dicom image in the same transverse position (Z-axis coordinate -729.125 mm). Fig. 9 Figure b in the figure is the tomographic image after posture adjustment. Fig. 9 As shown in Figure b, after the posture adjustment, the predicted target point 903 coincides with the target lesion 902, and the posture adjustment is completed.

[0163] In some embodiments, a same-layer posture adjustment mode can be set. When it is determined that the predicted target and the target lesion are located in the same fault layer, the user selects the same-layer posture adjustment mode during the automatic posture adjustment or manual posture adjustment process. At this time, only the same-layer angle can be changed during the posture adjustment process, and the cross-layer angle cannot be changed. The same-layer posture adjustment mode can avoid the problem of mistakenly crossing layers during the posture adjustment process by limiting the change of only the same-layer angle.

[0164] (2) Cross-layer posture adjustment.

[0165] Cross-layer posture adjustment means that the predicted target before and after posture adjustment has a cross-layer phenomenon in the transverse position (i.e., it is located on different tomographic images), the same-layer angle (left-right deflection angle) may remain unchanged or change, and the cross-layer angle (head-foot deflection angle) changes. Fig.10 As shown, in Fig.10 Figure a is the tomographic image before posture adjustment. Fig.10 The b figure in the figure is the tomographic image after posture adjustment. Fig.10 As shown in Figure a, there is no target lesion on the tomographic image where the predicted target point 1001 is located before the posture adjustment, so the predicted target point needs to be adjusted to the tomographic image showing the target lesion. In this case, cross-layer posture adjustment is required. After the cross-layer angle is changed, the Z-axis coordinate of the predicted target point 1002 is updated from -729.125mm to -727.875mm, and the predicted target point 1002 after the posture adjustment coincides with the target lesion 1003, completing the posture adjustment.

[0166] In some embodiments, a cross-layer posture adjustment mode can be set. When it is determined that the predicted target and the target lesion are located in different fault layers, the user selects the cross-layer posture adjustment mode during the automatic posture adjustment or manual posture adjustment process. At this time, only the cross-layer angle can be changed during the posture adjustment process, and the same-layer angle cannot be changed. The setting of the cross-layer posture adjustment mode allows the predicted target to be quickly adjusted to the fault layer where the target lesion is located during the posture adjustment process, thereby improving the posture adjustment efficiency.

[0167] It is understandable that after adjusting the predicted target point to the fault where the target lesion is located, if the predicted target point and the target lesion still cannot overlap, the same-layer posture adjustment mode can be selected at this time, and then the final posture adjustment can be completed in the same-layer posture adjustment mode.

[0168] (3) Free posture adjustment.

[0169] Free posture adjustment means that the doctor can control the robot's main hand to adjust the posture within the same layer and across layers at the same time. After the needle track posture changes, the predicted target position can change within the same layer or across layers. I will not go into details here.

[0170] After introducing the posture adjustment process, the triggering process of the medical imaging device in the embodiment of the present application is described below.

[0171] In some embodiments, the puncture assist device automatically controls the medical imaging device to perform a scanning operation.

[0172] Exemplarily, when the puncture auxiliary device automatically controls the medical imaging device to perform a scanning operation, when the auxiliary puncture device receives the prompt information of the surgical robot to perform the puncture operation, it controls the medical imaging device to perform the scanning operation. The puncture operation includes a posture adjustment operation or a needle insertion operation. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle. In this embodiment, the auxiliary puncture device can automatically control the medical imaging device to scan, which can improve the synchronization between the medical imaging device and the puncture and improve the real-time performance of the scan. The description of automatic scanning can refer to the description in the previous puncture system embodiment, which will not be repeated here.

[0173] After introducing the puncture method in the embodiment of the present application, the structure of the auxiliary puncture device and the surgical robot in the embodiment of the present application will be introduced in conjunction with the accompanying drawings.

[0174] See also Fig.11 FIG. 1 is a schematic diagram of the structure of an auxiliary puncture device 11 provided in one embodiment of the present application. Fig.11 As shown, the auxiliary puncture device in the embodiment of the present application includes a display module 1103 and a processor 1100, and the processor 1100 is used to execute the steps performed by the auxiliary puncture device in any of the above-mentioned puncture method embodiments.

[0175] In some other embodiments, the processor 1100 is used to perform the following steps: obtaining multiple tomographic images obtained by a medical imaging device performing a scanning operation on a puncture object; obtaining puncture status information of a puncture needle in a surgical robot, the puncture status information including status information of the puncture needle when the medical imaging device performs a scanning operation on the puncture object; determining position information of a needle tip point of the puncture needle and position information of a predicted target point based on the puncture status information, the predicted target point being: the spatial point reached by the needle tip point of the puncture needle after the puncture needle completes insertion according to the posture in the puncture status information; marking the needle tip point and the predicted target point on a first tomographic image based on the position information of the needle tip point and the position information of the predicted target point to obtain a marked first tomographic image; the first tomographic image includes at least one or two tomographic images among the multiple tomographic images; and displaying the marked first tomographic image through the display module 1103.

[0176] It should be understood that the processor 1100 may directly obtain multiple tomographic images from the medical imaging device, or may indirectly obtain multiple tomographic images from the medical imaging device; this application does not limit this. For example, the medical imaging device may directly report multiple tomographic images to the processor 1100, or the processor 1100 may actively obtain puncture status information from the medical imaging device; or the medical imaging device may send multiple tomographic images obtained by scanning to the transfer device, and the processor 1100 may obtain multiple tomographic images from the transfer device.

[0177] Similarly, the processor 1100 may directly obtain the puncture status information from the surgical robot, or may indirectly obtain the puncture status information from the surgical robot, and this application does not limit this. For example, the surgical robot may directly report the puncture status information to the processor 1100, or the processor 1100 may actively obtain the puncture status information from the surgical robot; or the surgical robot may also send the puncture status information to the transfer device, and the processor 1100 may obtain the puncture status information from the transfer device.

[0178] For example, Fig.11 As shown, in some other embodiments, the auxiliary puncture device also includes a memory 1101 and a computer program 1102 stored in the memory 1101 and executable on the processor 1100. When the processor 1100 executes the computer program 1102, the steps performed by the auxiliary puncture device in any of the above-mentioned puncture method embodiments are implemented.

[0179] In some other embodiments, the processor 1100 performs the following steps when executing the computer program 1102: obtaining multiple tomographic images obtained by a medical imaging device performing a scanning operation on a puncture object; obtaining puncture status information of a puncture needle in a surgical robot, the puncture status information including status information of the puncture needle when the medical imaging device performs a scanning operation on the puncture object; determining position information of the needle tip point of the puncture needle and position information of a predicted target point based on the puncture status information, the predicted target point being: the spatial point reached by the needle tip point of the puncture needle after the puncture needle completes the insertion according to the posture in the puncture status information; marking the needle tip point and the predicted target point on a first tomographic image based on the position information of the needle tip point and the position information of the predicted target point, to obtain a marked first tomographic image; the first tomographic image includes at least one or two tomographic images among the multiple tomographic images; and displaying the marked first tomographic image through the display module 1103.

[0180] Fig.11 This is merely an example of the auxiliary puncture device 11 and does not constitute a limitation on the auxiliary puncture device 11. The auxiliary puncture device 11 may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0181] The processor 1100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0182] In some embodiments, the memory 1101 may be an internal storage unit of the auxiliary puncture device 11, such as a hard disk or memory of the auxiliary puncture device 11. In other embodiments, the memory 1101 may also be an external storage device of the auxiliary puncture device 11, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the auxiliary puncture device 11. Further, the memory 1101 may also include both an internal storage unit of the auxiliary puncture device 11 and an external storage device. The memory 1101 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as the program code of the computer program, etc. The memory 1101 may also be used to temporarily store data that has been output or is to be output.

[0183] See also Fig.12 FIG. 1 is a schematic diagram of the structure of a surgical robot 12 provided in one embodiment of the present application. Fig.12 As shown, the auxiliary puncture device in the embodiment of the present application includes a robot master hand 1203, a robot slave hand 1204, and at least one processor 1200 ( Fig.12 Only one processor is shown in the figure), a memory 1201, and a computer program 1202 stored in the memory 1201 and executable on the at least one processor 1200, wherein the processor 1200 implements the steps performed by the first processor in the puncture system embodiment in any of the above-mentioned embodiments when executing the computer program 1202.

[0184] Fig.12This is merely an example of the surgical robot 12 and does not constitute a limitation of the surgical robot 12. The surgical robot 12 may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0185] The processor 1200 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0186] In some embodiments, the memory 1201 may be an internal storage unit of the surgical robot 12, such as a hard disk or memory of the surgical robot 12. In other embodiments, the memory 1201 may also be an external storage device of the surgical robot 12, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the surgical robot 12. Further, the memory 1201 may also include both an internal storage unit of the surgical robot 12 and an external storage device. The memory 1201 is used to store an operating system, an application program, a boot loader (Boot Loader), data, and other programs, such as the program code of the computer program. The memory 1201 may also be used to temporarily store data that has been output or is to be output.

[0187] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0188] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the steps performed by the auxiliary puncture device in the above-mentioned various puncture method embodiments, or the steps performed by the processor in the above-mentioned auxiliary puncture device embodiments.

[0189] An embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps performed by the auxiliary puncture device in the above-mentioned various puncture method embodiments, or the steps performed by the processor in the above-mentioned auxiliary puncture device embodiments.

[0190] The embodiment of the present application also provides a chip located in an electronic device, the chip comprising: a processing unit and a communication unit, the processing unit, for example, may be a processor, the communication unit, for example, may be an input / output interface, a pin or a circuit, etc. The processing unit may execute computer instructions to enable the electronic device to execute the steps executed by the auxiliary puncture device in any of the puncture methods provided in the embodiments of the present application, or the steps executed by the processor in the auxiliary puncture device embodiment.

[0191] Optionally, the computer instructions are stored in a storage unit.

[0192] Optionally, the storage unit is a storage unit within the chip, such as a register, a cache, etc. The storage unit can also be a storage unit located outside the chip within the terminal, such as a ROM or other types of static storage devices that can store static information and instructions, random RAM, etc. Among them, the processor mentioned in any of the above can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the above-mentioned feedback information transmission method. The processing unit and the storage unit can be decoupled and respectively set on different physical devices, and connected by wired or wireless means to implement the respective functions of the processing unit and the storage unit to support the system chip to implement the various functions in the above-mentioned embodiments. Alternatively, the processing unit and the memory can also be coupled on the same device.

[0193] Among them, the auxiliary puncture device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the projection device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0195] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0196] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0197] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0198] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0199] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An auxiliary puncture device, characterized in that: The auxiliary puncture device comprises: a display module and a processor, and the processor is used to perform the following steps: Acquiring multiple tomographic images obtained by a medical imaging device performing a scanning operation on the puncture object; Acquiring puncture status information of a puncture needle in the surgical robot, wherein the puncture status information includes status information of the puncture needle when the medical imaging device performs the scanning operation on the puncture object; According to the puncture state information, determine the position information of the needle tip of the puncture needle and the position information of the predicted target point, wherein the predicted target point is: the spatial point reached by the needle tip of the puncture needle after the puncture needle completes the insertion according to the posture in the puncture state information; According to the position information of the needle tip point and the position information of the predicted target point, the needle tip point and the predicted target point are marked on a first tomographic image to obtain a marked first tomographic image, wherein the first tomographic image includes at least one or two tomographic images among the multiple tomographic images; The marked first tomographic image is displayed by the display module.

2. The auxiliary puncture device according to claim 1, characterized in that: The puncture state information includes the position information of the needle tip, and the position information of the needle tip is calculated based on the relative position relationship between the puncture needle and the surgical robot. The processor further performs the following steps: extracting a needle track contour of the puncture needle according to the plurality of tomographic images, wherein the needle track contour represents a three-dimensional structure of the puncture needle; Determine the actual position information of the needle tip of the puncture needle according to the needle track contour, wherein the actual position information of the needle tip represents the position information of the needle tip of the puncture needle in the needle track contour; If it is determined that the needle track contour is abnormal based on the position information of the needle tip point and the actual position information of the needle tip point, a needle track abnormality warning is generated.

3. The auxiliary puncture device according to claim 1, characterized in that: The scanning operation is performed during the posture adjustment process, or the scanning operation is performed during the needle insertion process, the posture adjustment process includes adjusting the posture of the puncture needle, and the needle insertion process includes adjusting the puncture depth of the puncture needle.

4. The auxiliary puncture device according to claim 1, characterized in that: When the needle tip point and the predicted target point are marked on the same tomographic image, a line connecting the needle tip point and the predicted target point is also displayed on the marked first tomographic image.

5. The auxiliary puncture device according to claim 1, characterized in that: The processor also performs the following steps: While the marked first tomographic image is displayed by the display module, adjacent tomographic images of the first tomographic image are also displayed by the display module.

6. The auxiliary puncture device according to claim 1, characterized in that: The processor also performs the following steps: The display module also displays the puncture status information, which includes at least one of a puncture depth, a puncture speed and a posture angle. The puncture depth indicates the depth of the needle tip of the puncture needle after entering the skin. The posture angle indicates the angle of the puncture needle relative to the puncture object. The puncture speed indicates the moving speed of the puncture needle along the needle insertion direction.

7. The auxiliary puncture device according to any one of claims 1 to 6, characterized in that: The processor also performs the following steps: If a first posture adjustment instruction is received, first position information of a target lesion of the puncture object is identified from the multiple tomographic images, and the first posture adjustment instruction instructs the auxiliary puncture device to automatically adjust its posture; Obtaining first adjustment data according to the first position information and the puncture state information, wherein the first adjustment data represents posture change data of the puncture needle when the predicted target point is adjusted to a first position corresponding to the first position information; A first movement instruction corresponding to the first adjustment data is sent to the surgical robot, where the first movement instruction is used to control the surgical robot to move the puncture needle for posture adjustment.

8. The auxiliary puncture device according to claim 7, characterized in that: Before sending the first movement instruction corresponding to the first adjustment data to the surgical robot, the processor is configured to perform the following steps: If the processor determines that the puncture depth of the puncture needle is greater than a first threshold, or the distance between the predicted target point and the first position is greater than a second threshold, or direct posture adjustment based on the first adjustment data will cause the puncture needle to interfere with the target tissue organ of the puncture object, a second movement instruction is sent to the surgical robot, and the second movement instruction is used to control the surgical robot to move the puncture needle for needle retraction.

9. The auxiliary puncture device according to claim 7, characterized in that: Before sending a first movement instruction corresponding to the first adjustment data to the surgical robot, wherein the first movement instruction is used to control the surgical robot to move the puncture needle for posture adjustment, the processor further performs the following steps: Displaying the first adjustment data by the display module; If an instruction to adopt the first adjustment data is received, a first movement instruction corresponding to the first adjustment data is sent to the surgical robot.

10. The auxiliary puncture device according to any one of claims 1 to 6, characterized in that: The processor also performs the following steps: If a second posture adjustment instruction is received, a manual posture adjustment instruction is sent to the surgical robot, wherein the manual posture adjustment instruction is an instruction for the surgical robot to control the puncture needle to adjust its posture in response to a user operation; the second posture adjustment instruction instructs the user to select manual posture adjustment.

11. The auxiliary puncture device according to any one of claims 1 to 6, characterized in that: The scanning operation is performed by the auxiliary puncture device automatically controlling the medical imaging device. When the auxiliary puncture device automatically controls the medical imaging device to scan, the processor performs the following steps: If a prompt message is received for the surgical robot to perform a puncture operation, the medical imaging device is controlled to perform the scanning operation. The puncture operation includes a posture adjustment operation or a needle insertion operation. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

12. A puncture system, characterized in that: The puncture system comprises: a medical imaging device, a surgical robot and the auxiliary puncture device according to any one of claims 1 to 11, wherein: The medical imaging device is used to scan the puncture object to obtain a plurality of tomographic images, and send the plurality of tomographic images to the auxiliary puncture device; The surgical robot is used to send puncture status information of the puncture needle in the surgical robot to the auxiliary puncture device, wherein the puncture status information includes status information of the puncture needle when the medical imaging device performs the scanning operation on the puncture object; The auxiliary puncture device is used to determine the position information of the needle tip of the puncture needle and the position information of the predicted target point according to the puncture state information, and the predicted target point is: the spatial point reached by the needle tip of the puncture needle after the puncture needle completes the insertion according to the posture in the puncture state information; The auxiliary puncture device is used to mark the needle tip point and the predicted target point on a first tomographic image according to the position information of the needle tip point and the position information of the predicted target point, so as to obtain a marked first tomographic image; the first tomographic image includes at least one or two tomographic images of the multiple tomographic images; The auxiliary puncture device is used to display the annotated first tomographic image through a display module in the auxiliary puncture device.

13. The puncture system according to claim 12, characterized in that: The surgical robot comprises: a surgical robot master hand, a first processor and a surgical robot slave hand, the puncture needle is arranged on the surgical robot slave hand, and the surgical robot master hand is communicatively connected with the auxiliary puncture device; wherein: The main hand of the surgical robot is used to send the first puncture operation instruction to the first processor after receiving the first puncture operation instruction, and send prompt information for performing the puncture operation to the auxiliary puncture device; The first processor is configured to send a third movement instruction corresponding to the first puncture operation instruction to the slave hand of the surgical robot after receiving the first puncture operation instruction; The auxiliary puncture device is used to send a scanning instruction to the medical imaging device after receiving the prompt information; The medical imaging device is used to perform the scanning operation on the puncture object after receiving the scanning instruction; The surgical robot's slave hand is used to move the puncture needle according to the third movement instruction to perform a puncture operation corresponding to the first puncture operation instruction. The puncture operation includes a posture adjustment operation or a needle insertion operation. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

14. The puncture system according to claim 12, characterized in that: The surgical robot comprises: a surgical robot master hand, a first processor and a surgical robot slave hand, the puncture needle is arranged on the surgical robot slave hand, and the first processor is communicatively connected with the auxiliary puncture device; wherein: The surgical robot main hand is used to send the second puncture operation instruction to the first processor after receiving the second puncture operation instruction; The first processor is used to determine a fourth movement instruction corresponding to the second puncture operation instruction; The first processor is used to send a fourth movement instruction to the slave hand of the surgical robot, and send prompt information for performing a puncture operation to the auxiliary puncture device; The auxiliary puncture device is used to send a scanning instruction to the medical imaging device after receiving the prompt information; The medical imaging device is used to perform the scanning operation on the puncture object after receiving the scanning instruction; The surgical robot's slave hand is used to move the puncture needle according to the fourth movement instruction to perform a puncture operation corresponding to the second puncture operation instruction. The puncture operation includes a posture adjustment operation or a needle insertion operation. The posture adjustment operation is used to adjust the posture of the puncture needle, and the needle insertion operation is used to adjust the puncture depth of the puncture needle.

15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the computer program performs the steps performed by the processor in the puncture auxiliary device according to any one of claims 1 to 11.

Images