Bullseye plane display method and device, electronic equipment and storage medium

Abstract

This application provides a method, device, electronic device, and storage medium for displaying a target plane. The method includes: acquiring a first coordinate group of P markers attached to the patient's body surface in a CT coordinate system based on CT images; tracking a second coordinate group of the P markers in an optical coordinate system using an optical tracking device; registering the first and second coordinate groups to obtain a first real-time transformation matrix from the CT coordinate system to the optical coordinate system; calculating second real-time coordinates of the needle tip and needle tail in the CT coordinate system based on the first real-time transformation matrix, the first real-time needle tip coordinates, and the first real-time needle tail coordinates in the optical coordinate system; and displaying the positions of the needle tip, needle tail, and target point in the target plane based on the coordinates projected onto the target plane from the second real-time needle tip coordinates, the coordinates projected onto the target plane from the second real-time needle tail coordinates, and the target point coordinates in the CT coordinate system. This method can avoid harm to the patient caused by puncture surgery.

Description

Technical Field

[0001] This application relates to the field of surgical navigation technology, specifically to a method, device, electronic device, and storage medium for displaying a target plane. Background Technology

[0002] In traditional surgical navigation, it's crucial to determine the position of the needle tip relative to the target point (the selected point within the lesion in the human tissue). This is typically achieved by displaying the relative position of the needle tip and target point from a third-person perspective. However, a single perspective cannot accurately depict the relative position of the human tissue and the surgical needle, necessitating frequent switching of views to observe whether the needle is on the planned path. If the surgeon operates improperly and performs puncture before reaching the designated path, it could potentially harm the patient. Summary of the Invention

[0003] This application provides a method, device, electronic device, and storage medium for displaying the target plane, which can accurately display the positions of the target point, needle tip, and needle tail on the target plane, providing a target point-needle tip-needle tail view, facilitating observation of whether the needle tip of the surgical needle is on the puncture path, and avoiding situations where puncture is performed before reaching the designated path, causing harm to the patient.

[0004] A first aspect of this application provides a method for bullseye planar display, the method being applied to an electronic device in a bullseye planar display system, the bullseye planar display system including the electronic device, an optical tracking device, and a surgical needle; the method includes:

[0005] Acquire computed tomography (CT) images, and obtain the first coordinate set of P markers attached to the patient's body surface in the CT coordinate system based on the CT images, where P is an integer greater than or equal to 3;

[0006] The planned path is determined based on the target point and needle insertion point marked on the CT image, the first target point coordinates in the CT coordinate system are determined, and the first needle insertion point coordinates in the CT coordinate system are determined.

[0007] A target center plane is drawn with the first target point coordinates in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path.

[0008] The P markers are tracked in the second coordinate group of the optical coordinate system by an optical tracking device. The first coordinate group and the second coordinate group are registered to obtain the first real-time transformation matrix from the CT coordinate system to the optical coordinate system.

[0009] The first real-time transformation matrix and the first real-time needle tip coordinates of the surgical needle in the optical coordinate system are used to calculate the second real-time needle tip coordinates of the surgical needle in the CT coordinate system; the first real-time transformation matrix and the first real-time needle tail coordinates of the surgical needle in the optical coordinate system are used to calculate the second real-time needle tail coordinates of the surgical needle in the CT coordinate system.

[0010] The real-time needle tip coordinates and real-time needle tail coordinates are calculated based on the second real-time needle tip coordinates, the second real-time needle tail coordinates, and the target plane; the positions of the needle tip, the needle tail, and the target point are displayed on the target plane based on the real-time needle tip coordinates, the real-time needle tail coordinates, and the first target point coordinates.

[0011] Optionally, before calculating the second real-time needle tip coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinates in the optical coordinate system, the method further includes:

[0012] The first real-time needle tip coordinates in the optical coordinate system are calculated based on the real-time pose of the needle tip and the local coordinate matrix of the needle tip.

[0013] Before calculating the second real-time needle tail coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinates in the optical coordinate system, the method further includes:

[0014] The first real-time needle tail coordinates in the optical coordinate system are calculated based on the real-time pose of the needle tip and the local coordinate matrix of the needle tail.

[0015] Optionally, the target plane display system further includes a calibrator and a guide, the guide being located in a first local coordinate system. Before calculating the first real-time tip coordinates of the surgical needle in the optical coordinate system based on the real-time pose of the needle tip and the local coordinate matrix of the needle tip, and before calculating the first real-time tail coordinates of the surgical needle in the optical coordinate system based on the real-time pose of the needle tip and the local coordinate matrix of the needle tail, the method further includes:

[0016] The tip of the surgical needle is calibrated to obtain a second transformation matrix from the guide to the tip of the surgical needle;

[0017] The guide is tracked in real time by the optical tracking device to obtain the third real-time transformation matrix of the guide from the first local coordinate system to the optical coordinate system;

[0018] The real-time pose of the needle tip is calculated based on the second transformation matrix and the third real-time transformation matrix.

[0019] Optionally, the guide is provided with N markers, and the calibrator is provided with M markers, where M and N are both integers greater than or equal to 3. The calibrator is located in a second local coordinate system. The calibration of the tip of the surgical needle to obtain a second transformation matrix from the guide to the tip of the surgical needle includes:

[0020] The optical tracking device records the coordinates of N markers on the guide in the optical coordinate system, and the optical tracking device also records the coordinates of M markers on the calibrator in the optical coordinate system.

[0021] The third initial transformation matrix of the guide from the first local coordinate system to the optical coordinate system is determined based on the coordinates of the N markers in the first local coordinate system and the coordinates of the N markers in the optical coordinate system.

[0022] The fourth transformation matrix of the calibrator from the second local coordinate system to the optical coordinate system is determined based on the coordinates of the M markers in the second local coordinate system and the coordinates of the M markers in the optical coordinate system.

[0023] Based on the position of the tip of the surgical needle in the second local coordinate system, the translation matrix of the tip of the surgical needle relative to the origin of the second local coordinate system is obtained. The fourth transformation matrix is ​​multiplied by the translation matrix to obtain the fifth transformation matrix from the local coordinate system of the tip of the surgical needle with the tip of the surgical needle as the origin to the optical coordinate system.

[0024] A second transformation matrix is ​​obtained from the guide to the tip of the surgical needle based on the third initial transformation matrix and the fifth transformation matrix.

[0025] Optionally, the calculation of the real-time pose of the surgical needle tip based on the second transformation matrix and the third real-time transformation matrix includes:

[0026] Multiplying the third real-time transformation matrix by the inverse of the second transformation matrix yields the real-time pose of the surgical needle tip, which includes the translation and rotation of the local coordinate system of the surgical needle tip relative to the optical coordinate system.

[0027] Optionally, the step of calculating the second real-time needle tip coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinates in the optical coordinate system includes:

[0028] Multiply the inverse of the first real-time transformation matrix by the first real-time needle tip coordinates of the surgical needle in the optical coordinate system to obtain the second real-time needle tip coordinates of the surgical needle in the CT coordinate system.

[0029] The step of calculating the second real-time needle tail coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinates in the optical coordinate system includes:

[0030] Multiply the inverse of the first real-time transformation matrix by the first real-time needle tail coordinate of the surgical needle in the optical coordinate system to obtain the second real-time needle tip coordinate of the surgical needle in the CT coordinate system.

[0031] Optionally, after calculating the second real-time needle tip coordinates of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinates of the surgical needle in the optical coordinate system, and calculating the second real-time needle tail coordinates of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinates of the surgical needle in the optical coordinate system, the method further includes:

[0032] If the distance between the second real-time needle tip coordinate and the planned path is greater than a first threshold, or if the distance between the second real-time needle tail coordinate and the planned path is greater than the first threshold, a prompt message is generated to indicate that the surgical needle has deviated from the planned path.

[0033] A second aspect of this application provides a bullseye planar display device, the device being applied to an electronic device in a bullseye planar display system, the bullseye planar display system including the electronic device, an optical tracking device, and a surgical needle; the device includes:

[0034] The acquisition unit is used to acquire CT images and, based on the CT images, acquire the first coordinate set of P markers pasted on the patient's body surface in the CT coordinate system, where P is an integer greater than or equal to 3.

[0035] The determining unit is used to determine the planned path based on the target point and needle insertion point marked on the CT image, determine the first target point coordinates of the target point in the CT coordinate system, and determine the first needle insertion point coordinates of the needle insertion point in the CT coordinate system.

[0036] A drawing unit is used to draw a target center plane with the first target point coordinates in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path.

[0037] The registration unit is used to track the P markers in the second coordinate group in the optical coordinate system through the optical tracking device, and to perform registration according to the first coordinate group and the second coordinate group to obtain the first real-time transformation matrix from the CT coordinate system to the optical coordinate system.

[0038] The calculation unit is configured to calculate the second real-time needle tip coordinate of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinate of the surgical needle in the optical coordinate system; and to calculate the second real-time needle tail coordinate of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinate of the surgical needle in the optical coordinate system.

[0039] The calculation unit is also used to calculate the real-time needle tip projection coordinates and the real-time needle tail projection coordinates based on the second real-time needle tip coordinates, the second real-time needle tail coordinates and the target plane;

[0040] The display unit is used to display the position of the tip of the surgical needle, the position of the tail of the surgical needle, and the position of the target point on the target center plane according to the real-time needle tip projection coordinates, the real-time needle tail projection coordinates, and the first target point coordinates.

[0041] A third aspect of this application provides an electronic device including a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions to execute the step instructions as described in the first aspect of this application.

[0042] A fourth aspect of this application provides a computer-readable storage medium storing a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform some or all of the steps described in the first aspect of this application.

[0043] A fifth aspect of this application provides a computer program product, wherein the computer program product includes a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform some or all of the steps described in the first aspect of this application. The computer program product may be a software installation package.

[0044] In this embodiment, a computed tomography (CT) image is acquired. Based on the CT image, a first coordinate set of P markers attached to the patient's body surface in the CT coordinate system is obtained, where P is an integer greater than or equal to 3. A planned path is determined based on the target point and needle insertion point marked on the CT image. The first target point coordinates and the first needle insertion point coordinates in the CT coordinate system are determined. A target center plane is drawn with the first target point coordinates in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path. A second coordinate set of the P markers in the optical coordinate system is tracked using an optical tracking device. Registration is performed based on the first coordinate set and the second coordinate set to obtain a first coordinate system from the CT coordinate system to the optical coordinate system. A real-time transformation matrix is ​​used; based on the first real-time transformation matrix and the first real-time needle tip coordinates of the surgical needle in the optical coordinate system, the second real-time needle tip coordinates of the surgical needle in the CT coordinate system are calculated; based on the first real-time transformation matrix and the first real-time needle tail coordinates of the surgical needle in the optical coordinate system, the second real-time needle tail coordinates of the surgical needle in the CT coordinate system are calculated; based on the second real-time needle tip coordinates, the second real-time needle tail coordinates, and the target plane, the real-time needle tip projection coordinates and the real-time needle tail projection coordinates are calculated; based on the real-time needle tip projection coordinates, the real-time needle tail projection coordinates, and the first target point coordinates, the positions of the surgical needle tip, the surgical needle tail, and the target point are displayed on the target plane. In this embodiment, the positions of the target point, needle tip, and needle tail can be accurately displayed on the target plane, providing a target point-needle tip-needle tail perspective, facilitating observation of whether the needle tip is on the puncture path, and avoiding situations where puncture is performed before reaching the designated path, causing harm to the patient. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the structure of a bullseye planar display system provided in an embodiment of this application;

[0047] Figure 2 This is a schematic diagram of another target-oriented planar display system provided in an embodiment of this application;

[0048] Figure 3 This is a schematic flowchart of a target plane display method provided in an embodiment of this application;

[0049] Figure 4 This is a flowchart illustrating another method for displaying a bullseye planar image provided in an embodiment of this application;

[0050] Figure 5 This is a schematic diagram of the first coordinate group Ma in the CT coordinate system for five markers attached to the patient's body surface, as provided in an embodiment of this application.

[0051] Figure 6 This is a schematic diagram illustrating how to determine the target point, needle insertion point, and planned path on a CT image, as provided in an embodiment of this application.

[0052] Figure 7 This is a schematic diagram illustrating the positional relationship between the target center plane and the target point, provided in an embodiment of this application.

[0053] Figure 8 This is a schematic diagram illustrating the transformation relationship between the first local coordinate system of a guide and the second local coordinate system of a calibrator, provided in an embodiment of this application.

[0054] Figure 9 This is a schematic diagram of rigid registration of a first coordinate group Ma on a CT image with a second coordinate group Mb tracked by an optical device, provided in an embodiment of this application.

[0055] Figure 10 This is a schematic diagram illustrating the relative positional relationship of the needle tip, needle tail, and target point on the target center plane in a CT coordinate system, provided in an embodiment of this application.

[0056] Figure 11 This is a schematic diagram showing the relative positional relationship of the needle tip, needle tail, and target point on the target center plane in another CT coordinate system provided in this application embodiment;

[0057] Figure 12 This is a schematic diagram of the structure of a bullseye flat panel display device provided in an embodiment of this application;

[0058] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0059] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0061] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0062] The following description, in conjunction with the accompanying drawings, describes the target plane display method, device, electronic equipment, and storage medium of this application. It can accurately display the positions of the target point, needle tip, and needle tail on the target plane, providing a target point-needle tip-needle tail view. This facilitates observation of whether the tip of the surgical needle is on the puncture path, avoiding situations where puncture is performed before reaching the designated path, thus preventing harm to the patient.

[0063] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of a bullseye planar display system provided in an embodiment of this application. Figure 1 As shown, the bullseye planar display system 100 includes an electronic device 10, an optical tracking device 20, and a surgical needle 30;

[0064] The electronic device 10 is used to acquire computed tomography (CT) images and, based on the CT images, acquire the first coordinate set of P markers attached to the patient's body surface in the CT coordinate system, where P is an integer greater than or equal to 3.

[0065] The electronic device 10 is used to determine the planned path based on the target point and needle insertion point marked on the CT image, determine the first target point coordinates of the target point in the CT coordinate system, and determine the first needle insertion point coordinates of the needle insertion point in the CT coordinate system.

[0066] The electronic device 10 is used to draw a target center plane with the first target point coordinates in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path.

[0067] The electronic device 10 is used to track the P markers in a second coordinate group in an optical coordinate system using an optical tracking device, and to perform registration based on the first coordinate group and the second coordinate group to obtain a first real-time transformation matrix from the CT coordinate system to the optical coordinate system.

[0068] The electronic device 10 is used to calculate the second real-time needle tip coordinate of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinate of the surgical needle in the optical coordinate system; and to calculate the second real-time needle tail coordinate of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinate of the surgical needle in the optical coordinate system.

[0069] The electronic device 10 is used to calculate real-time needle tip projection coordinates and real-time needle tail projection coordinates based on the second real-time needle tip coordinates, the second real-time needle tail coordinates, and the target plane; and to display the position of the needle tip, the position of the needle tail, and the position of the target on the target plane based on the real-time needle tip projection coordinates, the real-time needle tail projection coordinates, and the first target point coordinates.

[0070] In this embodiment, the electronic device 10 and the optical tracking device 20 can be connected via a communication connection 70. This communication connection 70 can be a wired or wireless communication connection; this application does not limit the specific type of connection.

[0071] The electronic device 10 may acquire computed tomography (CT) images in any of the following ways: 1. By communicating with a CT device, where the CT device actively transmits its captured CT images to the electronic device 10, or the electronic device 10 sends a CT image acquisition request to the CT device, which responds by sending the CT image corresponding to the CT image number carried in the request to the electronic device 10. 2. The CT device uploads its captured CT images to a server, and the electronic device 10 retrieves the CT images from the server. 3. The electronic device acquires CT images from an external storage device (e.g., a portable hard drive).

[0072] The CT images in this application embodiment can be CT images of human tissues (e.g., lungs).

[0073] It should be noted that patients need to hold their breath during a CT scan, i.e., maintain their breathing. The patient's initial respiratory state during the CT scan can be recorded.

[0074] Before taking CT images of patient 60, the CT equipment can attach P markers (such as...) to the patient's body surface. Figure 1As shown in 41-4P), CT images can contain P markers pasted on the patient's body surface.

[0075] In this embodiment, the P markers can be optical markers (e.g., small balls that reflect near-infrared light). The P markers can be positioned within the tracking range of the optical tracking device 20. The surface of the markers can include a reflective coating for reflecting infrared light. The optical tracking device 20 can include a first infrared sensor 21 and a second infrared sensor 22. Both the first infrared sensor 21 and the second infrared sensor 22 can emit and receive infrared light. The markers (e.g., small balls that reflect near-infrared light) can be optical markers (e.g., small balls that reflect near-infrared light). Figure 1 Taking marker 41 as an example to illustrate the principle of optical tracking device 20 tracking markers, infrared (IR) light can be reflected (rather than scattered) back to the first infrared sensor 21 and the second infrared sensor 22 through a reflective coating on its surface. The first infrared sensor 21 and the second infrared sensor 22 utilize binocular vision to... Figure 1 The light rays shown (such as) Figure 1 The intersection of the dashed lines (as shown) allows for the measurement of the three-dimensional coordinates of the marker, thereby obtaining the three-dimensional coordinates of the marker in the optical coordinate system. The optical tracking device 20 can record the three-dimensional coordinates of the marker in real time. For example, the optical tracking device 20 can periodically record the three-dimensional coordinates of the marker.

[0076] After acquiring CT images, the electronic device 10 can obtain the first coordinate group of the P markers pasted on the patient's body surface in the CT coordinate system based on the CT images.

[0077] This application embodiment can display CT images on the screen of an electronic device. Doctors can manually mark the target point and needle insertion point on a cross-section of the CT image using display software installed on the electronic device. Based on the marked target point and needle insertion point on the cross-section of the CT image, the first target point coordinates in the CT coordinate system and the first needle insertion point coordinates in the CT coordinate system can be determined. A planned path can also be determined based on the marked target point and needle insertion point on the cross-section of the CT image. The planned path can be a straight line from the needle insertion point to the target point.

[0078] The entry point can be the point where the surgical needle 30 enters the body from the skin surface of the patient 60. The target point 50 can be located within the lesion area of ​​the patient 60. From Figure 1 As can be seen, the tail of the surgical needle 30 is fixed on the guide 80, and the tip of the surgical needle 30 can move toward the target point 50. After the surgical needle 30 reaches the target point, corresponding surgery (such as biopsy, ablation, drug injection, etc.) can be performed on the lesion area.

[0079] The bullseye plane in the CT coordinate system can be any shape, such as a circle, square, rectangle, or quadrilateral.

[0080] Because the target plane is perpendicular to the planned path, during the procedure, the real-time needle tip and tail projection coordinates can be calculated based on the second real-time needle tip coordinates in the CT coordinate system, the second real-time needle tail coordinates in the CT coordinate system, and the target plane. Based on these real-time needle tip and tail projection coordinates and the first target point coordinates in the CT coordinate system, the positions of the needle tip, tail, and target point are displayed on the target plane. Specifically, the real-time needle tip projection coordinates are the coordinates projected onto the target plane from the second real-time needle tip coordinates in the CT coordinate system, and the real-time needle tail projection coordinates are the coordinates projected onto the target plane from the second real-time needle tail coordinates in the CT coordinate system. This allows for accurate display of the target point, needle tip, and tail positions on the target plane, providing a target-tip-tail view for easy observation of whether the needle tip is on the puncture path, preventing injury to the patient due to puncture before reaching the designated path.

[0081] Please see Figure 2 , Figure 2 This is a schematic diagram of another target-oriented planar display system provided in an embodiment of this application. For example... Figure 2 As shown, the target center planar display system 100 also includes a guide 80 and a calibrator 90; the guide 80 is located in a first local coordinate system, and the calibrator 90 is located in a second local coordinate system. The guide 80 has N markers, and the calibrator has M markers, where M and N are both integers greater than or equal to 3.

[0082] The calibrator 90 may have a groove 91 on its surface, with an opening 911 at one end and a calibration surface 912 at the other end. The guide 80 may include a locking structure 81 and a first structure 82, which are fixedly connected. A second structure 92 may be provided on the surface of the calibrator 90. The first structure 82 may be located in a first local coordinate system, and the second structure 92 may be located in a second local coordinate system.

[0083] In order to accurately obtain the first real-time needle tip coordinates of the tip 32 of the surgical needle 30 in the optical coordinate system and the first real-time needle tail coordinates of the tail 31 of the surgical needle 30 in the optical coordinate system, it is first necessary to calibrate the tip 32 of the surgical needle 30 to obtain the second transformation matrix from the guide to the tip of the surgical needle.

[0084] During the surgery, when the optical tracking device 20 tracks the first structure 82 of the guide 80, a third real-time transformation matrix of the guide 80 from the first local coordinate system to the optical coordinate system is obtained; the real-time pose of the tip 32 of the surgical needle 30 is calculated based on the second transformation matrix and the third real-time transformation matrix. The first real-time coordinates of the tip 32 of the surgical needle 30 in the optical coordinate system can be calculated based on the real-time pose of the tip 32 and the tip 32 coordinate transformation matrix; the second real-time coordinates of the tip 32 of the surgical needle 30 in the CT coordinate system can be calculated based on the first real-time transformation matrix and the first real-time coordinates of the tip 32 of the surgical needle 30 in the optical coordinate system. The first real-time coordinates of the needle tail 31 of the surgical needle 30 in the optical coordinate system can be calculated based on the real-time pose of the needle tip 32 and the coordinate transformation matrix of the needle tail 31. The second real-time coordinates of the needle tail 31 of the surgical needle 30 in the CT coordinate system can also be calculated based on the first real-time transformation matrix and the first real-time coordinates of the needle tail 31 of the surgical needle 30 in the optical coordinate system.

[0085] Optionally, the first structure 82 is provided with N markers, and the second structure 92 is provided with M markers, where M and N are both integers greater than or equal to 3. Figure 2 Taking an example where both M and N are equal to 4. Figure 2 As shown, four markers (such as...) are set on the first structure 82. Figure 2 As shown in 821, 822, 823, and 824), four markers (such as...) are set on the second structure 92. Figure 2 921, 922, 923, 924 shown).

[0086] The calibration surface 912 may be parallel to the cross-section of the groove 91. The cross-section of the groove is V-shaped.

[0087] Since the tip 32 of the surgical needle 30 abuts against the calibration surface 912, the tail 31 of the surgical needle 30 is fixed by the locking mechanism 81, and the first structure 82 is fixedly connected to the locking mechanism 81. During the calibration process, the coordinates of the tip 32 of the surgical needle 30 in the second local coordinate system of the calibrator 90 are fixed, so the relative positional relationship of the tip 32 of the surgical needle 30 with respect to the first structure 82 will not change.

[0088] Whether during calibration or surgery, the guide 80 secures the needle tail 31 of the surgical needle 30 via the locking mechanism 81, and the fixed position remains unchanged.

[0089] The N markers on the first structure 82 can be optical markers (e.g., small spheres that reflect near-infrared light), and the M markers on the second structure 92 can also be optical markers (e.g., small spheres that reflect near-infrared light). The N markers on the first structure 82 can be arranged in a certain way (e.g., any three of the N markers are not in a straight line), and the M markers on the second structure 92 can also be arranged in a certain way (e.g., any three of the M markers are not in a straight line). It should be noted that the arrangement of the N markers on the first structure 82 and the M markers on the second structure 92 is different. For example, as... Figure 2 As shown, the main body of the first structure 82 can be a cross-shaped structure, with a base at the top of each cross (a total of 4 bases). An optical marker can be fixed on each base (a total of 4 optical markers). The cross structure is connected to the locking structure 81. The locking mechanism 81 can include a groove and at least two adjustable threads, which can be used to clamp surgical needles 30 of different radii.

[0090] like Figure 2 As shown, the main body of the second structure 92 can be a stainless steel plate. Four optical markers (reflecting near-infrared spheres) are fixed to the top of the plate by threads. One side of the plate has a groove (e.g., a conical groove), and at the end of the groove is a flat surface (calibration surface 912). The side of the plate has at least one conical verification hole (e.g., ...). Figure 2 As shown, there are three conical verification holes on the side of the steel plate. The conical verification holes on the side of the calibrator can be used to verify the accuracy of the calibration results.

[0091] In practical use, the end of the surgical needle to be punctured is first fixed with a guide. Then, the needle body is placed close to the calibrator, with the needle tip against the calibration surface at the end of the groove. An optical tracking device tracks the first and second structures. Then, a software algorithm can be used to calculate the transformation matrix between the first and second structures to determine the offset of the needle tip. A second transformation matrix from the guide 80 to the needle tip 32 of the surgical needle 30 can also be calculated using a software algorithm to determine the pose of the needle tip (the pose can include rotation and translation).

[0092] The optical tracking device 20 can be fixedly installed next to the guide 80, calibrator 90 and surgical needle 30.

[0093] Figure 2 The N and M markers in the diagram can be optical markers, and the surface of each marker can include a reflective coating that can be used to reflect infrared light. Figure 2The optical tracking device 20 and the electronic device 10 can communicate via a connection 70. This communication connection 70 can be a wired communication connection or a wireless communication connection, and this application does not limit it in this regard.

[0094] In this embodiment, the marker can be positioned within the tracking range of the optical tracking device 20. The marker can be an optical marker, and its surface may include a reflective coating for reflecting infrared light. The optical tracking device 20 may include a first infrared sensor 21 and a second infrared sensor 22. Both the first infrared sensor 21 and the second infrared sensor 22 can emit and receive infrared light. The marker (e.g.) Figure 2 Taking marker 921 as an example to illustrate the principle of optical tracking device 20 tracking markers, infrared (IR) light can be reflected (rather than scattered) back to the first infrared sensor 21 and the second infrared sensor 22 through a reflective coating on its surface. The first infrared sensor 21 and the second infrared sensor 22 utilize binocular vision to... Figure 2 The light rays shown (such as) Figure 2 The intersection of the dashed lines (as shown) allows for the measurement of the three-dimensional coordinates of the marker, thereby obtaining the three-dimensional coordinates of the marker in the optical coordinate system. The optical tracking device 20 can record the three-dimensional coordinates of the marker in real time. For example, the optical tracking device 20 can periodically record the three-dimensional coordinates of the marker.

[0095] Electronic device 10 can be a device with data processing and communication capabilities. For example, electronic device 10 can be a personal computer. Electronic device 10 can also include a display, which can display the target plane, the position of the tip of the surgical needle, the position of the tail of the surgical needle, and the position of the target point on the target plane. It can also display the real-time pose of the tip 32 of the surgical needle 30, and can display the coordinates of the tip 32 of the surgical needle 30 in the optical coordinate system.

[0096] In this embodiment, during the calibration process, the tip of the surgical needle rests against the calibration surface through a groove, ensuring that the surgical needle does not bend and fixing the position of the tip. This allows for the accurate acquisition of the second transformation matrix from the guide to the tip of the surgical needle. During the surgery, only the first structure of the guide needs to be tracked to accurately obtain the third real-time transformation matrix from the first local coordinate system to the optical coordinate system where the guide is located. The real-time pose of the tip of the surgical needle is accurately calculated using the second and third real-time transformation matrices, thereby accurately determining the position of the tip of the surgical needle.

[0097] Please see Figure 3 , Figure 3 This is a flowchart illustrating a target-center planar display method provided in an embodiment of this application. Figure 3As shown, this method can be applied to Figure 1 or Figure 2 The electronic device in the target-centric planar display system shown. The method may include the following steps.

[0098] 301. The electronic device acquires computed tomography (CT) images and obtains the first coordinate set of P markers attached to the patient's body surface in the CT coordinate system based on the CT images, where P is an integer greater than or equal to 3.

[0099] In this embodiment, after P markers are fixedly affixed to the patient's skin, the positions of the P markers on the patient's skin do not change. During CT image capture and subsequent surgical procedures, the positions of the P markers affixed to the patient's skin do not change. It should be noted that with the patient's breathing, the skin on the abdomen and chest may rise and fall, and the positions of the P markers may change accordingly.

[0100] Optionally, the electronic device may acquire the first coordinate set of the P markers attached to the patient's body surface in the CT coordinate system based on the CT image, which may include the following steps:

[0101] Electronic devices acquire CT images, which include P markers attached to the patient's body surface;

[0102] The electronic device performs at least one Gaussian difference processing on the CT image to obtain a Gaussian difference processed CT image;

[0103] The electronic device performs clustering processing on pixels in the CT image after Gaussian difference processing whose pixel values ​​are greater than or equal to a set threshold, to obtain Q first candidate coordinates. The set threshold is determined based on the maximum pixel value in the CT image after Gaussian difference processing. The clustering processing is a pixel distance-based clustering processing, where Q is an integer greater than or equal to P.

[0104] When Q equals P, the electronic device uses the Q first candidate coordinates as the first coordinate group of the P markers in the CT coordinate system.

[0105] Gaussian difference processing can increase the pixel values ​​of markers in CT images, making them larger than the pixel values ​​of human tissue (e.g., lungs). For example, a threshold can be set to be greater than 80% of the maximum pixel value in the CT image. Specifically, the threshold can be set to 90% of the maximum pixel value in the CT image. By setting this threshold, pixels with pixel values ​​greater than 90% in the Gaussian difference-processed CT image can be clustered based on pixel distance to obtain Q pixel sets (for example, when P markers are all spherical markers of the same size, the pixel distance between all pixels in each pixel set is less than 2d, where d is the diameter of the spherical marker). The average coordinates of each pixel set are used as the first candidate coordinates for combining pixels in that set, thus obtaining Q first candidate coordinates.

[0106] This application implements a method to increase the pixel values ​​of markers in CT images using Gaussian difference, and then calculates Q first candidate coordinates through pixel distance-based clustering. When Q equals P, the Q first candidate coordinates can be used as the first coordinate group of P markers in the CT coordinate system, thus obtaining the first coordinate group of P markers in the CT coordinate system. The first coordinate group includes the P coordinates of the P markers in the CT coordinate system.

[0107] 302. Determine the planned path based on the target point and needle entry point marked on the CT image, determine the first target point coordinates in the CT coordinate system, and determine the first needle entry point coordinates in the CT coordinate system.

[0108] This application embodiment can display CT images on the screen of an electronic device. Doctors can manually mark the target point and needle insertion point on a cross-section of the CT image using display software installed on the electronic device. Based on the marked target point and needle insertion point on the cross-section of the CT image, the first target point coordinates in the CT coordinate system and the first needle insertion point coordinates in the CT coordinate system can be determined. A planned path can also be determined based on the marked target point and needle insertion point on the cross-section of the CT image. The planned path can be a straight line from the needle insertion point to the target point.

[0109] The entry point can be the point where the surgical needle enters the body from the patient's skin surface. The target point can be located within the patient's lesion area.

[0110] 303. The electronic device draws the target center plane with the first target point coordinate in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path.

[0111] In the CT coordinate system, the target plane can be of any shape, such as a circle, square, rectangle, or quadrilateral. For example, the area of ​​the target plane can be set to X times the area of ​​the circular cross-section of the surgical needle. X can be a real number greater than 10. For example, X can be any value between 100 and 10000. For example, when the target plane is circular, its radius can be set to Y times the radius of the surgical needle. Y can be a real number greater than 5. For example, Y can be any value between 10 and 1000.

[0112] Steps 301 to 303 can be preparations before surgery.

[0113] 304. The electronic device tracks P markers in the second coordinate group of the optical coordinate system through the optical tracking device, and performs registration according to the first coordinate group and the second coordinate group to obtain the first real-time transformation matrix from the CT coordinate system to the optical coordinate system.

[0114] In this embodiment, registration can be performed based on the second coordinate group of P markers in the optical coordinate system and the first coordinate group of P markers in the CT coordinate system to obtain the first real-time transformation matrix from the CT coordinate system to the optical coordinate system.

[0115] Given the coordinates of at least three points in two coordinate systems, the transformation relationship between the two coordinate systems, i.e., the transformation matrix between them, can be obtained. Specifically, this can be calculated using the Singular Value Decomposition (SVD) algorithm. After the electronic device acquires the first set of coordinates of P markers in the CT coordinate system and the second set of coordinates of P markers in the optical coordinate system, it can calculate the first real-time transformation matrix from the CT coordinate system to the optical coordinate system based on the first set of coordinates of P markers in the CT coordinate system and the second set of coordinates of P markers in the optical coordinate system using the SVD algorithm.

[0116] 305. The electronic device calculates the second real-time needle tip coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinates in the optical coordinate system; and calculates the second real-time needle tail coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinates in the optical coordinate system.

[0117] Surgical needles can refer to various types of needles used in surgical navigation during clinical surgery. Examples include suture needles, puncture needles, and ablation needles. Puncture needles are used in puncture procedures. Puncture procedures are surgical methods for diagnosing or treating diseases. Under strict aseptic conditions, different specialized surgical needles are inserted into blood vessels, body cavities, or organs to aspirate fluid or tissue. Examining the aspirated fluid or tissue can reveal its nature and lesions, aiding in diagnosis. Medications can also be injected through the surgical needle to achieve therapeutic purposes. Puncture procedures can include: venous puncture, arterial puncture, lumbar puncture, thoracentesis, peritoneal puncture, pericardial puncture, bone marrow aspiration, liver puncture, spleen puncture, lung puncture, kidney puncture, cerebellomedullary cistern puncture, lymph node puncture, joint puncture, and maxillary sinus puncture.

[0118] 306. The electronic device calculates the real-time needle tip projection coordinates and the real-time needle tail projection coordinates based on the second real-time needle tip coordinates, the second real-time needle tail coordinates, and the target plane; and displays the position of the needle tip, the position of the needle tail, and the position of the target on the target plane based on the real-time needle tip projection coordinates, the real-time needle tail projection coordinates, and the first target point coordinates.

[0119] Among them, the real-time needle tip projection coordinates are the coordinates of the needle tip projected onto the target plane by the second real-time needle tip coordinates in the CT coordinate system, and the real-time needle tail projection coordinates are the coordinates of the needle tail projected onto the target plane by the second real-time needle tail coordinates in the CT coordinate system.

[0120] Steps 304 to 306 place the surgical needle within the tracking range of the optical tracking device, providing the surgeon or surgical robot with a guide to fix the surgical target and aim the puncture path for the patient. The surgeon or surgical robot can display the positions of the target point, needle tip, and needle tail on the target plane, making it easy to observe whether the needle tip is on the puncture path and avoiding situations where puncture is performed before reaching the designated path, which could cause harm to the patient.

[0121] In this embodiment, the positions of the target point, needle tip, and needle tail can be accurately displayed on the target plane, providing a target-needle tip-needle tail perspective, which facilitates observation of whether the needle tip of the surgical needle is on the puncture path, avoiding situations where puncture is performed before reaching the designated path, thus preventing harm to the patient.

[0122] Please see Figure 4 , Figure 4 This is a flowchart illustrating another target plane display method provided in an embodiment of this application. Figure 4 As shown, this method can be applied to Figure 2 The electronic device in the target-centric planar display system shown. The method may include the following steps.

[0123] 401. The electronic device acquires computed tomography (CT) images and obtains the first coordinate set of P markers attached to the patient's body surface in the CT coordinate system based on the CT images, where P is an integer greater than or equal to 3.

[0124] 402. Determine the planned path based on the target point and needle entry point marked on the CT image, determine the first target point coordinates in the CT coordinate system, and determine the first needle entry point coordinates in the CT coordinate system.

[0125] 403. The electronic device draws the target center plane with the first target point coordinate in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path.

[0126] 404. The electronic device tracks P markers in the second coordinate group of the optical coordinate system through the optical tracking device, and performs registration according to the first coordinate group and the second coordinate group to obtain the first real-time transformation matrix from the CT coordinate system to the optical coordinate system.

[0127] For details on the implementation of steps 401 to 404, please refer to [link to relevant documentation]. Figure 3 Steps 301 to 304 in the above steps will not be repeated here.

[0128] 405. The electronic device calibrates the tip of the surgical needle to obtain the second transformation matrix from the guide to the tip of the surgical needle; the guide is tracked in real time by an optical tracking device to obtain the third real-time transformation matrix of the guide from the first local coordinate system to the optical coordinate system; the real-time pose of the tip of the surgical needle is calculated based on the second transformation matrix and the third real-time transformation matrix.

[0129] In this embodiment of the application, before performing step 405, the tip of the surgical needle rests against the calibration surface through the groove to ensure that the surgical needle will not bend and to fix the position of the tip of the surgical needle.

[0130] Optionally, in step 405, the electronic device calibrates the tip of the surgical needle to obtain a second transformation matrix from the guide to the tip of the surgical needle, which may specifically include the following steps:

[0131] (11) The electronic device records the coordinates of N markers on the guide in the optical coordinate system through the optical tracking device, and records the coordinates of M markers on the calibrator in the optical coordinate system through the optical tracking device;

[0132] (12) The electronic device determines the third initial transformation matrix of the guide from the first local coordinate system to the optical coordinate system based on the coordinates of the N markers in the first local coordinate system and the coordinates of the N markers in the optical coordinate system;

[0133] (13) The electronic device determines the fourth transformation matrix of the calibrator from the second local coordinate system to the optical coordinate system based on the coordinates of the M markers in the second local coordinate system and the coordinates of the M markers in the optical coordinate system;

[0134] (14) The electronic device obtains the translation matrix of the tip of the surgical needle relative to the origin of the second local coordinate system based on the position of the tip of the surgical needle in the second local coordinate system, and multiplies the fourth transformation matrix with the translation matrix to obtain the fifth transformation matrix from the tip local coordinate system with the tip of the surgical needle as the origin to the optical coordinate system.

[0135] (15) The electronic device obtains a second transformation matrix from the guide to the tip of the surgical needle based on the third initial transformation matrix and the fifth transformation matrix.

[0136] In this embodiment, steps (11) to (15) are calibration processes. Because the arrangement of the N markers on the first structure of the guide (the relative positional relationship of the N markers) differs from the arrangement of the M markers on the second structure of the calibrator (the relative positional relationship of the M markers), the optical tracking device can track which markers are on the first structure and which are on the second structure.

[0137] In the embodiments of this application, the first local coordinate system, the second local coordinate system, and the optical coordinate system are all three-dimensional Cartesian coordinate systems. The first and second local coordinate systems can be defined by the user, and their definitions satisfy the right-hand rule.

[0138] Given the coordinates of at least three points in two coordinate systems, the transformation relationship between the two coordinate systems, i.e., the transformation matrix between them, can be obtained. Specifically, this can be calculated using the Singular Value Decomposition (SVD) algorithm. After the electronic device acquires the coordinates of N markers in the first local coordinate system and the coordinates of N markers in the optical coordinate system, it can calculate the third initial transformation matrix of the guide (first structure) from the first local coordinate system to the optical coordinate system based on the coordinates of the N markers in the first local coordinate system and the N markers in the optical coordinate system using the SVD algorithm.

[0139] After the electronic device obtains the coordinates of M markers in the second local coordinate system and the coordinates of M markers in the optical coordinate system, it can calculate the fourth transformation matrix of the calibrator (second structure) from the second local coordinate system to the optical coordinate system based on the coordinates of the M markers in the second local coordinate system and the coordinates of the M markers in the optical coordinate system according to the SVD algorithm.

[0140] For surgical needles of different radii, the position of the needle tip in the second local coordinate system is different.

[0141] The electronic device calculates the position of the needle tip in the second local coordinate system based on the radius of the surgical needle and the angle of the groove (if it is a V-shaped groove), thereby obtaining the local coordinate system of the needle tip (for example, it can be the local coordinate system of the needle tip established with the needle tip as the origin). Based on the position of the needle tip in the second local coordinate system and the fourth transformation matrix, the fifth transformation matrix from the local coordinate system of the needle tip to the optical coordinate system is obtained.

[0142] The third initial transformation matrix is ​​the initial transformation matrix of the guide (first structure) from the first local coordinate system to the optical coordinate system. The fifth transformation matrix is ​​the transformation matrix of the needle tip local coordinate system to the optical coordinate system. The transformation matrix from the guide (first structure) to the needle tip of the surgical needle, i.e., the second transformation matrix, can be obtained from the third initial transformation matrix and the fifth transformation matrix.

[0143] Optionally, step (15) may include the following steps:

[0144] The electronic device multiplies the inverse of the fifth transformation matrix with the third initial transformation matrix to obtain a second transformation matrix from the guide (first structure) to the tip of the surgical needle.

[0145] Optionally, in step 405, the electronic device tracks the guide in real time using the optical tracking device to obtain a third real-time transformation matrix of the guide from the first local coordinate system to the optical coordinate system. This may specifically include the following steps:

[0146] In this embodiment, the calibrator is removed during the surgery. The guide remains fixed to the tip of the surgical needle via a locking mechanism, and the second transformation matrix from the guide (first structure) to the tip of the surgical needle does not change. Because the position of the surgical needle tip changes during the surgery, the origin of the first local coordinate system where the guide (first structure) is located and the directions of the x, y, and z axes of the first local coordinate system change in the optical coordinate system, and the real-time transformation matrix from the first local coordinate system to the optical coordinate system also changes. To accurately determine the position of the surgical needle tip, a third real-time transformation matrix from the first local coordinate system to the optical coordinate system can be obtained. The real-time pose of the surgical needle tip is calculated based on the second and third real-time transformation matrices. The real-time pose of the surgical needle tip can include the translation and rotation of the surgical needle tip in the optical coordinate system, and the coordinates of the needle tip in the optical coordinate system can be determined based on the translation and rotation of the needle tip in the optical coordinate system.

[0147] The first local coordinate system can be a predefined coordinate system. The origin of the first local coordinate system can be chosen at any location within the first structure. The origin and the directions of the x, y, and z axes of the first local coordinate system can be pre-selected and defined, and the directions of the x, y, and z axes must satisfy the right-hand rule. Once the first local coordinate system is defined, the position of its origin and the directions of its x, y, and z axes will not change within the first local coordinate system.

[0148] The optical tracking device can periodically track the real-time transformation matrix from the first local coordinate system to the optical coordinate system. For example, every 0.05 seconds, based on the coordinates of N markers of the tracked first structure in the optical coordinate system and the coordinates of N markers in the first local coordinate system, the SVD algorithm is used to calculate the third real-time transformation matrix from the first local coordinate system to the optical coordinate system, and the third real-time transformation matrix can be updated every 0.05 seconds.

[0149] In this embodiment, during the calibration process, the tip of the surgical needle rests against the calibration surface through a groove, ensuring that the surgical needle does not bend and fixing the position of the tip. This allows for the accurate acquisition of the second transformation matrix from the guide (first structure) to the tip of the surgical needle. During the surgery, only the guide (first structure) needs to be tracked, allowing for the accurate acquisition of the third real-time transformation matrix from the first local coordinate system to the optical coordinate system where the guide (first structure) is located. The real-time pose of the tip of the surgical needle is accurately calculated using the second and third real-time transformation matrices, thereby accurately determining the position of the tip.

[0150] Optionally, in step 405, the electronic device calculates the real-time pose of the surgical needle tip based on the second transformation matrix and the third real-time transformation matrix, which may specifically include the following steps:

[0151] The electronic device multiplies the third real-time transformation matrix with the inverse of the second transformation matrix to obtain the real-time pose of the tip of the surgical needle. The real-time pose includes the translation and rotation of the local coordinate system of the tip of the surgical needle relative to the optical coordinate system.

[0152] Optionally, the calibrator has at least one verification hole on its side, which is used to verify the calibration results.

[0153] After calibration, select the verification hole from the at least one verification hole on the side of the calibrator that is closest in diameter to the surgical needle. Place the tip of the surgical needle in this verification hole and observe the degree of overlap between the displayed position of the verification hole and the position of the surgical needle tip on the electronic device's display. This verifies the calibration result. Generally, a higher degree of overlap indicates a better calibration result.

[0154] 406. The electronic device calculates the first real-time needle tip coordinates in the optical coordinate system based on the real-time pose of the needle tip and the local coordinate matrix of the needle tip, and calculates the first real-time needle tail coordinates in the optical coordinate system based on the real-time pose of the needle tip and the local coordinate matrix of the needle tail.

[0155] In this embodiment, after calibrating the tip of the surgical needle, since the origin of the local coordinate system of the needle tip is at the needle tip, and knowing the real-time pose of the needle tip, the translation component in the real-time pose of the needle tip can be used as the first real-time needle tip coordinate in the optical coordinate system. The first real-time needle tail coordinate in the optical coordinate system can be calculated based on the translation and rotation components in the real-time pose of the needle tip. Specifically, please refer to the following Pm... NDI and Pn NDI The calculation method.

[0156] The local coordinate matrix of the needle tip is the homogeneous coordinate representation of the needle tip in the local coordinate system of the needle tip. The local coordinate matrix of the needle tail is the homogeneous coordinate representation of the needle tail in the local coordinate system of the needle tip. The homogeneous coordinate representation is a 4*1 dimensional column matrix. When the last row is 1, the matrix represents a point in space; when the last dimension is 0, the matrix represents a vector in space.

[0157] 407. The electronic device calculates the second real-time needle tip coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinates in the optical coordinate system; and calculates the second real-time needle tail coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinates in the optical coordinate system.

[0158] Optionally, in step 407, the electronic device calculates the second real-time needle tip coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinates in the optical coordinate system. Specifically, this may include the following steps:

[0159] The electronic device multiplies the inverse of the first real-time transformation matrix with the first real-time needle tip coordinates of the surgical needle in the optical coordinate system to obtain the second real-time needle tip coordinates of the surgical needle in the CT coordinate system.

[0160] In step 407, the electronic device calculates the second real-time needle tail coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinates in the optical coordinate system. Specifically, this may include the following steps:

[0161] The electronic device multiplies the inverse of the first real-time transformation matrix with the first real-time needle tail coordinates of the surgical needle in the optical coordinate system to obtain the second real-time needle tip coordinates of the surgical needle tail in the CT coordinate system.

[0162] 408. The electronic device calculates the real-time needle tip projection coordinates and the real-time needle tail projection coordinates based on the second real-time needle tip coordinates, the second real-time needle tail coordinates, and the target plane; and displays the position of the needle tip, the position of the needle tail, and the position of the target on the target plane based on the real-time needle tip projection coordinates, the real-time needle tail projection coordinates, and the first target point coordinates.

[0163] For details on the implementation of steps 407 to 408, please refer to [link to relevant documentation]. Figure 3 Steps 305 to 306 in the above steps will not be repeated here.

[0164] Optionally, after performing step 407, the following steps may also be performed:

[0165] If the distance between the second real-time needle tip coordinate and the planned path is greater than a first threshold, or if the distance between the second real-time needle tail coordinate and the planned path is greater than the first threshold, the electronic device generates a prompt message to indicate that the surgical needle has deviated from the planned path.

[0166] In this embodiment, the first threshold can be preset and stored in the memory of the electronic device (e.g., non-volatile memory). The first threshold can be set to at least 0.1 times the diameter of the surgical needle. For example, the first threshold can be set to 1 times the diameter of the surgical needle.

[0167] Optionally, after performing step 408, the following steps may also be performed:

[0168] If the distance between the real-time needle tip projection coordinates and the first target point coordinates is greater than a first threshold, or if the distance between the real-time needle tail projection coordinates and the first target point coordinates is greater than the first threshold, the electronic device generates a prompt message to indicate that the surgical needle has deviated from the planned path.

[0169] Optionally, the first threshold can also be determined based on the size of the lesion area. For example, if the lesion area is large in the CT image, the first threshold can be set relatively large, selecting the target point as the center of the lesion area. Even if the tip or tail of the surgical needle is far from the planned path, it can still ensure that the surgical needle can enter the lesion area, guaranteeing the surgical effect. If the lesion area is small in the CT image, the first threshold can be set relatively small, preventing the surgical needle from deviating from the lesion area.

[0170] The prompt information can be at least one of voice prompts, text prompts, or image prompts.

[0171] The method described in this application can be used in puncture surgery. It accurately represents the relative positions of the target point, needle tip, and needle tail during puncture surgery, facilitating puncture under the guidance of a puncture surgical robot or physician.

[0172] The following is combined Figures 5 to 11 The following describes the specific process of bullseye plane display in embodiments of this application. In these embodiments, P=5, M=N=4 are used as examples, and a puncture surgery is taken as an example. The bullseye plane display method in embodiments of this application may specifically include the following steps.

[0173] 1. Acquire the patient's CT image, reconstruct it, segment the markers, and obtain the first coordinate group Ma of the 5 markers pasted on the patient's body surface in the CT coordinate system.

[0174] Please see Figure 5 , Figure 5 This is a schematic diagram of the first coordinate group Ma in the CT coordinate system for five markers pasted on the patient's body surface, as provided in an embodiment of this application.

[0175] 2. Perform path planning on the CT image, determine the planned path, and generate the target point coordinates and needle insertion point coordinates in the CT coordinate system, respectively, Pt CT and Pe CT .

[0176] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating how to determine the target point, needle insertion point, and planned path on a CT image, as provided in an embodiment of this application. The straight-line path between the needle insertion point and the target point can be used as the planned path.

[0177] 3. Set the target point Pt in the CT coordinate system CT Using the center point as the vector from the target point to the needle insertion point as the normal vector, the aiming plane is drawn and denoted as the target center plane. Taking a circular plane as an example, the target center plane is perpendicular to the planned path in the CT coordinate system, and the small circular ball at the center of the target center plane is the target point.

[0178] Please see Figure 7 , Figure 7 This is a schematic diagram illustrating the positional relationship between the bullseye plane and the target point, provided in an embodiment of this application.

[0179] 4. Surgical needle calibration: Obtain the second transformation matrix T2 from the guide to the tip of the surgical needle; this matrix represents the transformation relationship between the first local coordinate system of the guide and the second local coordinate system of the calibrator. Where b 11 -b33 Let T be the rotation component of the second transformation matrix T2. x2 T y2 T z2 This is the translation component of the second transformation matrix T2, with the last row fixed as [0 0 01].

[0180]

[0181] Please see Figure 8 , Figure 8 This is a schematic diagram illustrating the transformation relationship between the first local coordinate system of a guide and the second local coordinate system of a calibrator, as provided in an embodiment of this application.

[0182] Steps 1 through 4 are part of the pre-operative preparation process.

[0183] 5. Using an optical tracking device (an optical system mounted on the optical tracking device), five markers attached to the patient's body surface are tracked to obtain a second coordinate group Mb. The first coordinate group Ma on the CT image is rigidly registered with the second coordinate group Mb tracked by the optical device to obtain a first real-time transformation matrix T1. This first real-time transformation matrix T1 represents the transformation relationship (including translation and rotation) between the CT coordinate system and the optical coordinate system, where a... 11 -a 33 Let T be the rotation component of the first real-time transformation matrix T1. x1 T y1 T z1 This is the translation component of the first real-time transformation matrix T1.

[0184]

[0185] Please see Figure 9 , Figure 9 This is a schematic diagram of a rigid registration of a first coordinate group Ma on a CT image with a second coordinate group Mb tracked by an optical device, provided in an embodiment of this application.

[0186] 6. The guide is tracked in real time using an optical tracking device, and the third real-time transformation matrix T3 between the guide and the first local coordinate system is obtained. The third real-time transformation matrix T3 represents the transformation relationship between the guide and the optical system coordinate system. Where c... 11 -c 33 T is the rotation component of the third real-time transformation matrix T3. x3 T y3 T z3 This is the translation component of the third real-time transformation matrix T3.

[0187]

[0188] The real-time pose tip of the needle is calculated from this, and is represented as:

[0189] Tip = T3 * T2 -1

[0190] Multiplying the third real-time transformation matrix T3 by the inverse of the second transformation matrix T2 yields the real-time pose of the surgical needle tip.

[0191] 7. Calculate the first real-time needle tip coordinates Pm in the optical coordinate system. NDI The first real-time needle tail coordinate Pn in the optical coordinate system of the surgical needle tail. NDI When the length of the puncture needle is l, since the needle tip is located at the origin of the calibrator's local coordinate system after calibration, and the needle tail is located at the -z axis of the calibrator's local coordinate system, Pm NDI and Pn NDI Calculate them as follows:

[0192]

[0193]

[0194] Since the calibrated needle tip is located at the origin of the local coordinate system of the needle tip (the origin of the local coordinate system of the calibrator after translation), the first real-time needle tip coordinate Pm in the optical coordinate system can be calculated according to the above formula. NDI The first real-time needle tail coordinate Pn in the optical coordinate system of the surgical needle tail. NDI .in, It is the local coordinate matrix of the needle tip, which represents a point in space. It is the local coordinate matrix of the needle tail, which represents a point in space.

[0195] 8. Register the needle tip coordinates and needle tail coordinates to the CT coordinate system using T1, and obtain the second real-time needle tip coordinates Pm in the CT coordinate system. CT The second real-time needle tail coordinates Pn in the CT coordinate system. CT :

[0196] Pm CT =T1 -1 *Pm NDI

[0197] Pn CT =T1 -1 *Pn NDI

[0198] The inverse of the first real-time transformation matrix T1 is compared with the first real-time needle tip coordinate Pm in the optical coordinate system. NDIMultiplying these values ​​yields the second real-time needle tip coordinates Pm in the CT coordinate system. CT The inverse of the first real-time transformation matrix T1 is compared with the first real-time needle tail coordinate Pn in the optical coordinate system. NDI Multiplying these values ​​yields the second real-time needle tip coordinates Pn of the surgical needle tail in the CT coordinate system. CT .

[0199] 9. Convert the needle tip coordinates (Pm) in the CT coordinate system. CT ) and needle tail coordinates (Pn) CT Projecting the needle tip onto the target center plane yields the projected needle tip coordinates (Pm). proj ) and the projected coordinates of the needle tail (Pn) proj On the target plane, draw small balls representing the needle tip and the needle tail (the needle tip and needle tail can be distinguished by different colors). The projected coordinates of the needle tip and the needle tail can change in real time. The projected coordinates of the needle tip are equivalent to the real-time projected coordinates of the needle tip mentioned above, and the projected coordinates of the needle tail are equivalent to the real-time projected coordinates of the needle tail mentioned above. The calculation method for the projected coordinates is as follows:

[0200] Taking the needle tip coordinate as an example, let the needle tip coordinate (Pm) in the CT coordinate system be... CT ), the projected pointer coordinates (Pm) proj They are respectively:

[0201] Pm CT =(x0,y0,z0)

[0202] Pm proj =(x p ,y p ,z p )

[0203] Let the plane equation of the target center plane be:

[0204] Ax + By + Cz + D = 0

[0205] Based on the vertical constraint, the projected coordinates can be calculated as follows:

[0206]

[0207]

[0208]

[0209] Similarly, the projected coordinates of the needle tail can be calculated.

[0210] Please see Figure 10 , Figure 10This is a schematic diagram illustrating the relative positional relationship of the needle tip, needle tail, and target point on the target center plane in a CT coordinate system, provided in an embodiment of this application. Figure 10 It can be seen that the needle tip and needle tail are far from the target point.

[0211] 10. During actual puncture guidance, the doctor observes the view of the target plane to see if the small ball representing the needle tip and the small ball representing the needle tail are located at the center of the disk in the target plane. If not, the posture of the surgical needle is adjusted, and steps 5-9 are repeated until the small ball representing the needle tip and the small ball representing the needle tail are exactly located at the center of the disk in the target plane. This indicates that the puncture needle body coincides with the puncture path, and a safe puncture can be performed at this time.

[0212] Please see Figure 11 , Figure 11 This is a schematic diagram illustrating the relative positional relationship of the needle tip, needle tail, and target point on the target center plane in another CT coordinate system provided in this application embodiment. From Figure 11 As can be seen, the needle tip and needle tail are relatively close to the target point, indicating that the needle body coincides with the puncture path, and a safe puncture can be performed at this time.

[0213] The above describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments provided herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0214] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0215] Please see Figure 12 , Figure 12This is a schematic diagram of a bullseye planar display device provided in an embodiment of this application. The bullseye planar display device 1200 is used in an electronic device within a bullseye planar display system. The bullseye planar display system includes the electronic device, an optical tracking device, and a surgical needle. The device includes:

[0216] The acquisition unit 1201 is used to acquire CT images and acquire the first coordinate group of P markers pasted on the patient's body surface in the CT coordinate system based on the CT images, where P is an integer greater than or equal to 3.

[0217] The determining unit 1202 is used to determine the planned path based on the target point and needle insertion point marked on the CT image, determine the first target point coordinates of the target point in the CT coordinate system, and determine the first needle insertion point coordinates of the needle insertion point in the CT coordinate system.

[0218] The drawing unit 1203 is used to draw a target center plane with the first target point coordinates in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path.

[0219] The registration unit 1204 is used to track the P markers in the second coordinate group in the optical coordinate system through the optical tracking device, and perform registration according to the first coordinate group and the second coordinate group to obtain the first real-time transformation matrix from the CT coordinate system to the optical coordinate system.

[0220] The calculation unit 1205 is used to calculate the second real-time needle tip coordinate of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinate of the surgical needle in the optical coordinate system; and to calculate the second real-time needle tail coordinate of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinate of the surgical needle in the optical coordinate system.

[0221] The calculation unit 1205 is also used to calculate the real-time needle tip projection coordinates and the real-time needle tail projection coordinates based on the second real-time needle tip coordinates, the second real-time needle tail coordinates and the target plane;

[0222] Display unit 1206 is used to display the position of the tip of the surgical needle, the position of the tail of the surgical needle, and the position of the target point on the target center plane according to the real-time needle tip projection coordinates, the real-time needle tail projection coordinates, and the first target point coordinates.

[0223] Optionally, the calculation unit 1205 is further configured to calculate the first real-time needle tip coordinates of the surgical needle tip in the optical coordinate system based on the real-time pose of the needle tip and the local coordinate matrix of the needle tip.

[0224] The calculation unit 1205 is also used to calculate the first real-time needle tail coordinates of the needle tail in the optical coordinate system based on the real-time pose of the needle tip and the local coordinate matrix of the needle tail.

[0225] Optionally, the target plane display system further includes a calibrator and a guide, the guide being located in a first local coordinate system, and the target plane display device 1200 may also include a calibration unit 1207;

[0226] The calibration unit 1207 is used to calibrate the tip of the surgical needle to obtain a second conversion matrix from the guide to the tip of the surgical needle;

[0227] The computing unit 1205 is further configured to track the guide in real time through the optical tracking device, obtain the third real-time transformation matrix of the guide from the first local coordinate system to the optical coordinate system, and calculate the real-time pose of the tip of the surgical needle based on the second transformation matrix and the third real-time transformation matrix.

[0228] Optionally, the guide has N markers, and the calibrator has M markers, where M and N are both integers greater than or equal to 3. The calibrator is located in a second local coordinate system.

[0229] The calibration unit 1207 calibrates the tip of the surgical needle to obtain a second transformation matrix from the guide to the tip of the surgical needle, including: recording the coordinates of N markers on the guide in the optical coordinate system using the optical tracking device; recording the coordinates of M markers on the calibrator in the optical coordinate system using the optical tracking device; determining a third initial transformation matrix of the guide from the first local coordinate system to the optical coordinate system based on the coordinates of the N markers in the first local coordinate system and the coordinates of the N markers in the optical coordinate system; and determining a third initial transformation matrix of the guide from the first local coordinate system to the optical coordinate system based on the coordinates of the M markers in the second local coordinate system. The coordinates in the local coordinate system and the coordinates of the M markers in the optical coordinate system determine the fourth transformation matrix of the calibrator from the second local coordinate system to the optical coordinate system; the translation matrix of the tip of the surgical needle relative to the origin of the second local coordinate system is obtained based on the position of the tip of the surgical needle in the second local coordinate system, and the fourth transformation matrix is ​​multiplied by the translation matrix to obtain the fifth transformation matrix from the tip local coordinate system to the optical coordinate system with the tip of the surgical needle as the origin; the second transformation matrix from the guide to the tip of the surgical needle is obtained based on the third initial transformation matrix and the fifth transformation matrix.

[0230] Optionally, the calculation unit 1205 calculates the real-time pose of the surgical needle tip based on the second transformation matrix and the third real-time transformation matrix, including:

[0231] Multiplying the third real-time transformation matrix by the inverse of the second transformation matrix yields the real-time pose of the surgical needle tip, which includes the translation and rotation of the local coordinate system of the surgical needle tip relative to the optical coordinate system.

[0232] Optionally, the calculation unit 1205 calculates the second real-time needle tip coordinates of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinates of the surgical needle in the optical coordinate system, including: multiplying the inverse matrix of the first real-time transformation matrix with the first real-time needle tip coordinates of the surgical needle in the optical coordinate system to obtain the second real-time needle tip coordinates of the surgical needle in the CT coordinate system;

[0233] The calculation unit 1205 calculates the second real-time needle tail coordinate of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinate of the surgical needle in the optical coordinate system, including: multiplying the inverse matrix of the first real-time transformation matrix with the first real-time needle tail coordinate of the surgical needle in the optical coordinate system to obtain the second real-time needle tip coordinate of the surgical needle in the CT coordinate system.

[0234] Optionally, the target flat display device 1200 may also include a prompting unit 1208;

[0235] The prompting unit 1208 is used to generate prompting information when the distance between the second real-time needle tip coordinate and the planned path is greater than a first threshold or the distance between the second real-time needle tail coordinate and the planned path is greater than the first threshold. The prompting information is used to prompt that the surgical needle has deviated from the planned path.

[0236] In this embodiment, the determining unit 1202, drawing unit 1203, registration unit 1204, calculation unit 1205, and calibration unit 1207 can be processors in an electronic device. The display unit 1206 can be a display in an electronic device. The acquisition unit 1201 can be a communication module in an electronic device. The prompting unit 1208 can be a speaker or display in an electronic device.

[0237] In this embodiment, the positions of the target point, needle tip, and needle tail can be accurately displayed on the target plane, providing a target-needle tip-needle tail perspective, which facilitates observation of whether the needle tip of the surgical needle is on the puncture path, avoiding situations where puncture is performed before reaching the designated path, thus preventing harm to the patient.

[0238] Please see Figure 13 , Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 13 As shown, the electronic device 1300 includes a processor 1301 and a memory 1302, which are interconnected via a communication bus 1303. The communication bus 1303 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 1303 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 13 The bus is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. Memory 1302 stores computer programs, which include program instructions. Processor 1301 is configured to invoke these program instructions, which include instructions for execution. Figure 3 or Figure 4 It includes some or all of the steps in the methods.

[0239] Processor 1301 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the above scheme.

[0240] The memory 1302 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0241] The electronic device 1300 may further include a communication module 1304 and a display 1305. The communication module 1304 can communicate with the optical tracking device. The communication module 1304 can be a wireless communication module (e.g., a WiFi module, a Bluetooth module, etc.) or a wired communication module.

[0242] In addition, the electronic device 1300 may also include general components such as communication interfaces (e.g., USB interfaces, microphone interfaces, etc.) and antennas, which will not be described in detail here.

[0243] In this embodiment, the positions of the target point, needle tip, and needle tail can be accurately displayed on the target plane, providing a target-needle tip-needle tail perspective, which facilitates observation of whether the needle tip of the surgical needle is on the puncture path, avoiding situations where puncture is performed before reaching the designated path, thus preventing harm to the patient.

[0244] This application also provides a computer-readable storage medium storing a computer program for electronic data interchange that causes a computer to perform some or all of the steps of any of the target plane display methods described in the above method embodiments.

[0245] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0246] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0247] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0248] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0249] Furthermore, the functional units in the various embodiments of the application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software program module.

[0250] If the integrated unit is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0251] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage device, which may include: a flash drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0252] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A bullseye planar display device, characterized by, The device is used in an electronic device within a bullseye planar display system, the bullseye planar display system including the electronic device, an optical tracking device, and a surgical needle; the device includes: The acquisition unit is used to acquire CT images and, based on the CT images, acquire the first coordinate set of P markers pasted on the patient's body surface in the CT coordinate system, where P is an integer greater than or equal to 3. The determining unit is used to determine the planned path based on the target point and needle insertion point marked on the CT image, determine the first target point coordinates of the target point in the CT coordinate system, and determine the first needle insertion point coordinates of the needle insertion point in the CT coordinate system. A drawing unit is used to draw a target center plane with the first target point coordinates in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path. The registration unit is used to track the P markers in the second coordinate group in the optical coordinate system through the optical tracking device, and to perform registration according to the first coordinate group and the second coordinate group to obtain the first real-time transformation matrix from the CT coordinate system to the optical coordinate system. The calculation unit is configured to calculate the second real-time needle tip coordinate of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tip coordinate of the surgical needle in the optical coordinate system; and to calculate the second real-time needle tail coordinate of the surgical needle in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinate of the surgical needle in the optical coordinate system. The calculation unit is also used to calculate the real-time needle tip projection coordinates and the real-time needle tail projection coordinates based on the second real-time needle tip coordinates, the second real-time needle tail coordinates and the target plane; The display unit is used to display the position of the tip of the surgical needle, the position of the tail of the surgical needle, and the position of the target point on the target center plane according to the real-time needle tip projection coordinates, the real-time needle tail projection coordinates, and the first target point coordinates.

2. The bullseye planar display device of claim 1, wherein, The calculation unit is also used to calculate the first real-time needle tip coordinates of the surgical needle in the optical coordinate system based on the real-time pose of the needle tip and the local coordinate matrix of the needle tip. The calculation unit is also used to calculate the first real-time needle tail coordinates of the surgical needle tail in the optical coordinate system based on the real-time pose of the needle tip and the local coordinate matrix of the needle tail.

3. The bullseye planar display device of claim 2, wherein, The bullseye planar display system further includes a calibrator and a guide, the guide being located in a first local coordinate system; the bullseye planar display device further includes a calibration unit; The calibration unit is used to calibrate the tip of the surgical needle to obtain a second conversion matrix from the guide to the tip of the surgical needle; The computing unit is also used to track the guide in real time through the optical tracking device and obtain the third real-time transformation matrix of the guide from the first local coordinate system to the optical coordinate system; The calculation unit is also used to calculate the real-time pose of the tip of the surgical needle based on the second transformation matrix and the third real-time transformation matrix.

4. The bullseye planar display apparatus of claim 3, wherein, The guide has N markers, and the calibrator has M markers, where M and N are both integers greater than or equal to 3. The calibrator is located in a second local coordinate system. The calibration unit is also used for: The optical tracking device records the coordinates of N markers on the guide in the optical coordinate system, and the optical tracking device also records the coordinates of M markers on the calibrator in the optical coordinate system. The third initial transformation matrix of the guide from the first local coordinate system to the optical coordinate system is determined based on the coordinates of the N markers in the first local coordinate system and the coordinates of the N markers in the optical coordinate system. The fourth transformation matrix of the calibrator from the second local coordinate system to the optical coordinate system is determined based on the coordinates of the M markers in the second local coordinate system and the coordinates of the M markers in the optical coordinate system. The fifth transformation matrix from the local coordinate system of the surgical needle tip to the optical coordinate system is obtained based on the position of the needle tip in the second local coordinate system and the fourth transformation matrix. A second transformation matrix is ​​obtained from the guide to the tip of the surgical needle based on the third initial transformation matrix and the fifth transformation matrix.

5. The bullseye planar display apparatus of claim 3, wherein, The computing unit is also used for: Multiplying the third real-time transformation matrix by the inverse of the second transformation matrix yields the real-time pose of the surgical needle tip, which includes the translation and rotation of the local coordinate system of the surgical needle tip relative to the optical coordinate system.

6. The bullseye planar display apparatus of any one of claims 1-5, wherein, The computing unit is also used for: Multiply the inverse of the first real-time transformation matrix by the first real-time needle tip coordinates of the surgical needle in the optical coordinate system to obtain the second real-time needle tip coordinates of the surgical needle in the CT coordinate system. The step of calculating the second real-time needle tail coordinates in the CT coordinate system based on the first real-time transformation matrix and the first real-time needle tail coordinates in the optical coordinate system includes: Multiply the inverse of the first real-time transformation matrix by the first real-time needle tail coordinates of the surgical needle in the optical coordinate system to obtain the second real-time needle tip coordinates of the surgical needle tail in the CT coordinate system.

7. The bullseye planar display device according to any one of claims 1 to 5, characterized in that, The bullseye flat display device also includes a prompting unit; The prompting unit is used to generate prompting information when the distance between the second real-time needle tip coordinate and the planned path is greater than a first threshold or the distance between the second real-time needle tail coordinate and the planned path is greater than the first threshold. The prompting information is used to indicate that the surgical needle has deviated from the planned path.

8. An electronic device, characterized in that, The system includes a processor and a memory, the memory being used to store a computer program, the computer program including program instructions, and the processor being configured to invoke the program instructions to execute a bullseye planar display method, the bullseye planar display method being applied to an electronic device in a bullseye planar display system, the bullseye planar display system including the electronic device, an optical tracking device, and a surgical needle; The target center plane display method includes: Acquire computed tomography (CT) images, and obtain the first coordinate set of P markers attached to the patient's body surface in the CT coordinate system based on the CT images, where P is an integer greater than or equal to 3; The planned path is determined based on the target point and needle insertion point marked on the CT image, the first target point coordinates in the CT coordinate system are determined, and the first needle insertion point coordinates in the CT coordinate system are determined. A target center plane is drawn with the first target point coordinates in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path. The P markers are tracked in the second coordinate group of the optical coordinate system by an optical tracking device. The first coordinate group and the second coordinate group are registered to obtain the first real-time transformation matrix from the CT coordinate system to the optical coordinate system. The first real-time transformation matrix and the first real-time needle tip coordinates of the surgical needle in the optical coordinate system are used to calculate the second real-time needle tip coordinates of the surgical needle in the CT coordinate system; the first real-time transformation matrix and the first real-time needle tail coordinates of the surgical needle in the optical coordinate system are used to calculate the second real-time needle tail coordinates of the surgical needle in the CT coordinate system. The real-time needle tip coordinates and real-time needle tail coordinates are calculated based on the second real-time needle tip coordinates, the second real-time needle tail coordinates, and the target plane; the positions of the needle tip, the needle tail, and the target point are displayed on the target plane based on the real-time needle tip coordinates, the real-time needle tail coordinates, and the first target point coordinates.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to execute a bullseye planar display method. The bullseye planar display method is applied to an electronic device in a bullseye planar display system, the bullseye planar display system including the electronic device, an optical tracking device, and a surgical needle. The target center plane display method includes: Acquire computed tomography (CT) images, and obtain the first coordinate set of P markers attached to the patient's body surface in the CT coordinate system based on the CT images, where P is an integer greater than or equal to 3; The planned path is determined based on the target point and needle insertion point marked on the CT image, the first target point coordinates in the CT coordinate system are determined, and the first needle insertion point coordinates in the CT coordinate system are determined. A target center plane is drawn with the first target point coordinates in the CT coordinate system as the center point, and the target center plane is perpendicular to the planned path. The P markers are tracked in the second coordinate group of the optical coordinate system by an optical tracking device. The first coordinate group and the second coordinate group are registered to obtain the first real-time transformation matrix from the CT coordinate system to the optical coordinate system. The first real-time transformation matrix and the first real-time needle tip coordinates of the surgical needle in the optical coordinate system are used to calculate the second real-time needle tip coordinates of the surgical needle in the CT coordinate system; the first real-time transformation matrix and the first real-time needle tail coordinates of the surgical needle in the optical coordinate system are used to calculate the second real-time needle tail coordinates of the surgical needle in the CT coordinate system. The real-time needle tip coordinates and real-time needle tail coordinates are calculated based on the second real-time needle tip coordinates, the second real-time needle tail coordinates, and the target plane; the positions of the needle tip, the needle tail, and the target point are displayed on the target plane based on the real-time needle tip coordinates, the real-time needle tail coordinates, and the first target point coordinates.

Images