Surgical needle positioning display method and related device

By using reference plane and projection point assistance in percutaneous puncture surgery, the problem of surgical needle aligning the virtual puncture path in three-dimensional space is solved, improving operational efficiency and accuracy.

CN115500940BActive Publication Date: 2024-11-01SHENZHEN WEIDE PRECISION MEDICAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211001558.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-19
Publication Date
2024-11-01
Estimated Expiration
2042-08-19

AI Technical Summary

Technical Problem

In percutaneous puncture surgery, since the puncture path is only a virtual path, it is difficult for doctors to accurately grasp the adjustment direction and distance of the surgical needle in three-dimensional real space, resulting in a long alignment process of overlapping the surgical needle with the puncture path.

Method used

By obtaining the puncture path, a reference plane and a first intersection point with which the angle is greater than or equal to the first threshold is determined, and a projection point of the reference plane, the first intersection point and the surgical needle are displayed to provide a sense of direction and help the operator move the surgical needle to Reduce alignment time.

Benefits of technology

It significantly reduces the time of the overlapping alignment process between the surgical needle and the puncture path, improves the puncture efficiency and reduces the difficulty of operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115500940B_ABST
    Figure CN115500940B_ABST
Patent Text Reader

Abstract

The embodiment of the present application provides a method and related device for positioning and displaying a surgical needle, wherein the method includes: obtaining a puncture path, which is a virtual path for the surgical needle to pierce a target point; determining a reference plane, the angle between the reference plane and the puncture path is greater than or equal to a first threshold, and intersects with the puncture path at a first intersection; displaying the reference plane and the first intersection; determining a first projection point of the first reference point on the reference plane, and determining a second projection point of the second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle; and displaying the first projection point and the second projection point. The present application can reduce the time spent on the alignment process of aligning the surgical needle with the puncture path.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and specifically to a positioning display method and related devices for a surgical needle. Background Art

[0002] With the continuous development of computer science and technology, medical technology has also ushered in major breakthroughs. For example, before percutaneous puncture surgery, the doctor can determine the puncture path of the surgical needle according to the specific situation of the object, and the puncture path can be understood as a virtual path. During the operation, the optical tracking device can track the surgical needle in real time and display it on the display device. The doctor can use the surgical needle and puncture path displayed on the display device as a reference to let the surgical needle in the surgical space coincide with the puncture path for effective puncture.

[0003] However, since the puncture path is only a virtual path displayed by the display device and does not exist in the real space; and the actual space is three-dimensional, it is difficult for doctors and other operators to grasp the adjustment direction and distance of the surgical needle, resulting in the alignment process of aligning the surgical needle with the puncture path taking a lot of time. Summary of the invention

[0004] The embodiments of the present application provide a positioning display method and related devices for a surgical needle. Through the present application, the time spent on the alignment process of aligning the surgical needle with the puncture path can be reduced, thereby improving the puncture efficiency.

[0005] In a first aspect, an embodiment of the present application provides a method for displaying the positioning of a surgical needle, comprising:

[0006] Obtaining a puncture path, where the puncture path is a virtual path for the surgical needle to puncture the target point;

[0007] Determine a reference plane, wherein the angle between the reference plane and the puncture path is greater than or equal to a first threshold, and the reference plane intersects the puncture path at a first intersection point;

[0008] displaying the reference plane and the first intersection point;

[0009] Determine a first projection point of the first reference point on the reference plane, and determine a second projection point of the second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle;

[0010] The first projection point and the second projection point are displayed.

[0011] The method provided in the embodiment of the present application obtains the virtual puncture path of the surgical needle, determines a reference plane whose angle with the puncture path is greater than or equal to a first threshold value according to the puncture path, displays the reference plane and the first intersection point between the reference plane and the puncture path; when displaying the surgical needle, displays the first projection point of the first reference point on the reference plane, and displays the second projection point of the second reference point on the reference plane. Since the ultimate goal of the surgical needle is to align with the puncture path, the present application can allow the operator to know the position of the surgical needle more clearly and intuitively, such as the inclination angle of the surgical needle, by displaying the reference plane, the first intersection point and the projection point of the reference point on the reference plane. If the distance between the first projection point and the second projection point is too far, it can be considered that the inclination angle of the surgical needle is too large; it can also allow the operator to know the azimuth distance between the surgical needle and the puncture path more clearly and intuitively, such as the surgical needle is on the right side of the puncture path, etc. Therefore, the above method can allow the operator to have a sense of direction when moving the surgical needle. The operator uses the first intersection point as the target point and makes the first projection point and the second projection point coincide with the first intersection point as much as possible, which means that the surgical needle and the puncture path are roughly aligned. The position of the surgical needle can be fine-tuned subsequently to complete the alignment of the puncture path, greatly reducing the user's operating difficulty and further reducing the time spent on the alignment process of aligning the surgical needle with the puncture path.

[0012] In particular, when the puncture path is perpendicular to the reference plane, the surgical needle can be moved so that the first projection point, the second projection point and the first intersection point coincide with each other, and the surgical needle can be considered to be aligned with the puncture path.

[0013] In combination with the first aspect, in a possible implementation manner, before determining a first projection point of the first reference point on the reference plane and determining a second projection point of the second reference point on the reference plane, the method further includes:

[0014] Determine a projection direction vector of a reference direction vector on the reference plane, wherein the reference direction vector is a fixed direction determined according to the real space of the surgical needle;

[0015] Display the first auxiliary line corresponding to the above projection direction vector.

[0016] In combination with the first aspect, in a possible implementation manner, determining the reference plane, displaying the reference plane and the first intersection point, and displaying the first auxiliary line corresponding to the projection direction vector include:

[0017] The coordinate plane of the space coordinate system where the puncture path is located is taken as a first plane, wherein the first plane includes a first candidate point and a first candidate line;

[0018] Determine an orientation transformation matrix and a rotation angle, wherein the orientation transformation matrix is ​​used to rotate the first plane to a position where the angle between the first plane and the puncture path is greater than or equal to the first threshold value and the first candidate point is located on the puncture path; the rotation angle is the angle between the direction vector of the direction of the first candidate line and the projection direction vector;

[0019] In the coordinate plane, the first plane is rotated based on the rotation angle to obtain a second plane;

[0020] The second plane is transformed based on the orientation transformation matrix to obtain the reference plane, the first intersection point and the first auxiliary line; the first auxiliary line is obtained by rotating and translating the first candidate line; the first intersection point is obtained by translating the first candidate point;

[0021] The reference plane, the first intersection point and the first auxiliary line are displayed.

[0022] In combination with the first aspect, in a possible implementation, the surgical needle is clamped by a clamp, at least three markers are fixedly arranged on the clamp, and the spatial coordinates of the first reference point and the second reference point are determined by collecting the coordinates of the at least three markers by an optical tracking device; the reference direction vector includes any one or more of the following:

[0023] The direction vector of the target point pointing to the direction of the optical tracking device, the direction vector of the target point pointing to the direction of the feet of the implementation object, and the direction vector of the target point pointing to the direction of the head of the implementation object.

[0024] In combination with the first aspect, in a possible implementation manner, the first intersection point is the target point.

[0025] In combination with the first aspect, in a possible implementation manner, the method further includes:

[0026] A second auxiliary line is displayed, wherein the second auxiliary line includes at least one concentric circle auxiliary line with the first intersection as the center, and a plurality of angle auxiliary lines on the concentric circle auxiliary line.

[0027] In combination with the first aspect, in a possible implementation, the first reference point is the needle tail of the surgical needle, and the second reference point is the needle tip of the surgical needle, or is a rotation point for controlling the surgical needle.

[0028] In a second aspect, an embodiment of the present application provides a positioning display device for a surgical needle, comprising:

[0029] An acquisition unit, used for acquiring a puncture path, where the puncture path is a virtual path of the surgical needle puncturing the target point;

[0030] a determination unit, configured to determine a reference plane, wherein an angle between the reference plane and the puncture path is greater than or equal to a first threshold value, and the reference plane intersects the puncture path at a first intersection point;

[0031] A display unit, used for displaying the reference plane and the first intersection point;

[0032] The determination unit is further used to determine a first projection point of the first reference point on the reference plane, and to determine a second projection point of the second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle;

[0033] The display unit is also used to display the first projection point and the second projection point.

[0034] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a display screen, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions so that the method in the first aspect or any possible implementation manner of the first aspect is executed.

[0035] In a fourth aspect, an embodiment of the present application provides a chip, comprising a logic circuit and an interface, wherein the logic circuit and the interface are coupled; the interface is used to input and / or output code instructions, and the logic circuit is used to execute the code instructions so that the method in the first aspect or any possible implementation of the first aspect is executed.

[0036] In a fifth aspect, an embodiment of the present application discloses a computer program product, which includes program instructions. When the program instructions are executed by a processor, the method in the first aspect or any possible implementation of the first aspect is executed.

[0037] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is run on a processor, the method in the first aspect or any possible implementation of the first aspect is executed. Exemplarily, the computer program product may be a software installation package. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the following is a brief introduction to the drawings required for use in the embodiments of the present application or the background technology.

[0039] Figure 1 is a schematic diagram of a surgical scene based on a robotic arm provided in an embodiment of the present application;

[0040] Figure 2 is a schematic diagram of a surgical needle provided in an embodiment of the present application;

[0041] Figure 3 is a schematic diagram of a sight provided in an embodiment of the present application;

[0042] Figure 4 is a schematic diagram of a scenario for determining the direction of a sight according to the field of view direction of an optical tracking device provided by an embodiment of the present application;

[0043] Figure 5 It is a schematic diagram of the relationship between a vector and a 90° direction line and a 0° direction line provided in an embodiment of the present application;

[0044] Figure 6a It is a schematic diagram of a scenario of finding a puncture path according to a sight provided in an embodiment of the present application;

[0045] Figure 6b is a schematic diagram of determining the coordinates of a reference point on a surgical needle provided in an embodiment of the present application;

[0046] Figure 7 This is a schematic diagram of a result of searching for a puncture path according to an aimer provided in an embodiment of the present application;

[0047] Figure 8 It is a flow chart of a method for displaying the positioning of a surgical needle provided in an embodiment of the present application;

[0048] Fig. 9 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0049] Fig.10 It is a schematic diagram of the structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear contrary indication in its context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items. The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order.

[0051] At present, with the continuous development of science and technology, the application of robotic arms in surgery is becoming more and more extensive. In the embodiment of the present application, the robotic arm can be understood as a device for fixing the surgical needle in space. For ease of understanding, the example of a doctor performing percutaneous renal surgery on a patient is used for introduction. Figure 1 , Figure 1 This is a schematic diagram of a surgical scene based on a robotic arm provided in an embodiment of the present application.

[0052] like Figure 1 The surgical scene shown includes a patient 101, a robotic arm 103, a surgical needle 104, an optical tracking device 105, and a display device 106. The display device 106 is in communication with the optical tracking device 105, and the surgical needle 104 is clamped by a clamp, which includes a marker, and the marker is within the tracking range of the optical tracking device 105.

[0053] like Figure 1 As shown, the robotic arm 103 and the surgical needle 104 are connected together through a physical structure. Exemplarily, the connection between the robotic arm 103 and the surgical needle 104 can be a fixed connection, such as welding; or it can be a movable connection.

[0054] A marker is connected to the surgical needle 104, and the surface of the marker includes a reflective coating. In the embodiment of the present application, the marker is located within the tracking range of the optical tracking device, and the optical tracking device 105 can determine the spatial position information of the marker through the reflective coating on the surface of the marker to achieve real-time tracking of the marker. It can be understood that since the structure of the surgical needle 104 is fixed, the relative relationship between other positions in the surgical needle 104 and the marker is fixed. Therefore, the spatial position information of other positions on the surgical needle can be obtained based on the spatial position information of the marker and the relative position relationship with the marker, for example, the spatial position information of the needle tip of the surgical needle 104 can be obtained.

[0055] To facilitate understanding of the specific structure of the surgical needle 104, for example, reference may be made to Figure 2 , Figure 2 Schematic diagram of a surgical needle provided in an embodiment of the present application. Figure 2 As shown, the surgical needle includes a needle body 200 and a needle handle 201, and a holder 202 is connected to the needle handle, specifically, by a threaded connection capable of adjusting the tightness. The holder 202 exemplarily includes four markers, namely, marker 203, marker 204, marker 205, and marker 206. It is understood that the number of markers can be adjusted in actual situations, as long as the number of markers is greater than or equal to 3.

[0056] In the embodiment of the present application, the surgical needle 104 is connected to the robotic arm 103, and the doctor can move the position of the surgical needle 104 by adjusting the robotic arm 103. In addition, the optical tracking device 105 can track the surgical needle 104 in real time, and then display the position of the surgical needle 104 in real space in real time through the display device 106.

[0057] During the operation, the patient 101 lies on the operating table, and the doctor can plan the puncture path 102 of the surgical needle 104 in advance according to the location of the lesion of the patient 101. It can be understood that in the actual space, the surgical needle 104 is positioned by the optical tracking device 105, that is, the spatial position of the surgical needle 104 is based on the world coordinate system corresponding to the optical tracking device 105. Therefore, the puncture path 102 planned by the doctor should also be a puncture path in the world coordinate system, so that the surgical needle can have the conditions to "coincide" with the puncture path 102.

[0058] For example, the patient's kidney can be scanned by an ultrasound device for collecting ultrasound images, and the ultrasound probe of the ultrasound device can include a marker for positioning. Similar to the surgical needle 104, since the position of the marker on the ultrasound probe is fixed, the spatial position information of the lesion can be determined by the relative position relationship between the scanning range of the ultrasound device and the ultrasound probe and the spatial position relationship of the marker, and then the puncture path is determined based on the spatial position information of the lesion.

[0059] In the above manner, the spatial position of the puncture path and the spatial position of the surgical needle are both obtained based on the tracking of the optical tracking device 105 , thereby achieving spatial unification of the puncture path 102 and the surgical needle 104 .

[0060] In the embodiment of the present application, the puncture path 102 is not a path existing in the real space, but should be understood as the target path of the real surgical needle 104. Figure 1 As shown, the puncture path 102 includes a target point 1021 and a needle insertion point 1022. The target point 1021 can be understood as a lesion or other target position. During the operation, the doctor will use the robot arm 103 to adjust the direction of the real surgical needle 104 as parallel to the direction of the puncture path 102 as much as possible, and point the surgical needle 104 to the target point 1021 as much as possible, and then slowly advance the surgical needle 104 for puncture.

[0061] It is understandable that the puncture path 102 can be determined in combination with various information, such as the size, position, distance from the skin, and relationship between the target point 1021 and surrounding organs. It is understandable that after the puncture path 102 is determined, the position of the needle entry point 1022 where the surgical needle 104 punctures the skin of the patient 101 can be obtained.

[0062] It should also be understood that if the patient 101 does not change position, the puncture path displayed by the display device 106 will not change position. However, in the process of moving the surgical needle 104 in real space to align the puncture path, the position of the surgical needle displayed by the display device 106 will continue to change.

[0063] At present, during the operation, doctors and other operators can move the surgical needle in real space according to the relative position relationship between the puncture path and the surgical needle displayed in real time on the display device, so as to align the planned puncture path for puncture. However, since the puncture path is only a virtual path displayed by the display device and does not exist in the real space; and the actual space is three-dimensional, it is difficult for the operator to grasp the adjustment direction and distance of the surgical needle, resulting in a lot of time spent on the alignment process of aligning the surgical needle with the puncture path.

[0064] Based on the above problems, the embodiments of the present application provide a surgical needle positioning method and related devices. It is understandable that the surgical needle positioning method provided in the embodiments of the present application can be performed by a surgical needle positioning device, and the surgical needle positioning device can be any device that can implement the method provided in the present application. For example, the surgical needle positioning device can be an electronic device such as a laptop or desktop computer. It should be understood that the method embodiments of the present application can also be implemented by a processor executing a computer program code.

[0065] In order to understand the method provided by the present application vividly, the following is an example of a doctor performing percutaneous renal surgery on a patient to introduce the method provided by the present application. Exemplarily, the method provided by the present application includes the following steps:

[0066] 1. CT image acquisition preparation

[0067] The doctor determines whether it is necessary to acquire enhanced CT images based on the patient's specific condition. If the doctor determines that enhanced CT images are not necessary, the patient is asked to lie on the CT scanning bed and then pushed into the CT scanning window to acquire ordinary CT images. If the doctor determines that the patient needs to acquire enhanced CT images, contrast agent is injected into the patient's vein and then pushed into the CT scanning window to acquire enhanced CT images.

[0068] It is understandable that contrast agents can be developed in areas with rich blood vessels through the blood circulation system. The development of these areas can be figuratively understood as enhancement. The CT images obtained by CT scanning after enhancement can obtain more information about the kidneys, such as kidney stones and masses, kidney cysts, etc. It is understandable that compared with ordinary CT images, the subsequent registration of enhanced CT images with ultrasound images can also allow doctors to obtain more kidney information.

[0069] 2. CT data acquisition

[0070] The patient is pushed into the CT scanning window, and CT data is collected according to the judgment result in step 1, wherein the CT data includes a CT image. After the scanning is completed, the CT image data of the patient's kidney is obtained.

[0071] It can be understood that if an enhanced CT scan is performed on a patient, the enhanced CT scan includes plain scan phase, arterial phase, venous phase and delayed phase scans. Generally, the plain scan phase data can be used for subsequent steps.

[0072] 3. CT data processing

[0073] The CT image collected in step 2 is input into the surgical needle positioning device (also referred to as a renal surgery navigation system) provided in the present application. In the renal surgery navigation system, a three-dimensional renal contour is segmented and reconstructed from the CT data collected in step 2 above by an artificial intelligence algorithm. For ease of understanding and description, the three-dimensional renal contour is referred to as CT-renal contour.

[0074] Exemplarily, the artificial intelligence algorithm may be a deep learning algorithm, stochastic gradient descent (SGD), backward propagation, forward propagation, etc. It is understandable that the CT-renal contour may include the entire kidney information, such as the outer contour of the kidney, the internal substance of the kidney, etc.

[0075] 4. Ultrasound image acquisition

[0076] During the operation, the patient lies prone on the operating table, and the ultrasound device is used to scan and observe the patient's kidneys along the direction of the patient's back spine, and the real-time spatial position of the ultrasound image collected by the ultrasound device is located in combination with the optical tracking device. It is understandable that during the ultrasound image acquisition process, the patient can be reminded to keep the breathing rate as slow and stable as possible to reduce the spatial position fluctuation error caused by breathing.

[0077] 5. Ultrasonic image processing

[0078] In the renal surgery navigation system, an artificial intelligence algorithm is used to segment and reconstruct the three-dimensional renal contour based on the multiple ultrasound images collected in the above step 4 and the corresponding spatial position information. For ease of understanding and description, the three-dimensional renal contour is referred to as the ultrasound-renal contour.

[0079] 6. Registration

[0080] In this step, the renal surgery navigation system registers the CT-renal contour and the ultrasound-renal contour.

[0081] The registration between CT-kidney contour and ultrasound-kidney contour can be implemented by an iterative closest point (ICP) algorithm, which can transform point sets in different coordinate systems into a common coordinate system by minimizing the registration error.

[0082] Exemplarily, the above-mentioned registration process can be divided into two stages: coarse registration and fine registration. The coarse registration stage can be understood as a relatively rough registration performed when the transformation between the source point cloud and the target point cloud is almost unknown, and its main purpose is to provide a better initial value of the transformation matrix for fine registration; fine registration can be understood as a given initial transformation matrix, and further optimization to obtain a more accurate transformation matrix, that is, the final registration matrix.

[0083] 7. Determine the puncture path

[0084] For example, the kidney can be reconstructed in three dimensions based on the CT image of the kidney, and then the doctor determines the target point and the needle insertion point according to the patient's condition, for example, according to the position of the lesion inside the patient, the size of the lesion, the distance between the lesion and the skin, etc. In the embodiment of the present application, the puncture path can be understood as a straight line or line segment including the target point and the needle insertion point. Generally, the puncture path is a line segment, and one of the endpoints is the target point.

[0085] 8. Position the surgical needle according to the aiming surface

[0086] 8.1. Draw the sight

[0087] In the embodiment of the present application, the aiming device can be understood as an auxiliary line (such as a line segment or a curve, etc.) used to assist the surgical needle in positioning, and the aiming device can be displayed by a display device. In the embodiment of the present application, the aiming device can be considered as virtual and should not be understood as a device in the real space.

[0088] It is understandable that the aiming device in the present application needs to be perpendicular to the puncture path in space, and the center of the aiming device needs to coincide with the target point of the puncture path. However, it is difficult to determine the aiming device by only one plane and one point in space. Therefore, the aiming device can be drawn on the XOY plane first, and then the aiming device can be transformed in position according to the spatial position information of the puncture path.

[0089] In the embodiment of the present application, the aiming plane can be understood as a plane perpendicular to the puncture path. It can be understood that there are multiple planes perpendicular to the puncture path in space, among which the aiming plane including the target point of the puncture path can be understood as the target aiming plane. In the embodiment of the present application, according to the relationship between the point and the plane, the target point of the puncture path is included in the target aiming plane, and it can be understood that the spatial coordinates corresponding to the target point are on the target aiming plane.

[0090] 8.1.1. Draw the sight on the XOY plane

[0091] For easier understanding, see Figure 3 , Figure 3 It is a schematic diagram of a sight provided in an embodiment of the present application.

[0092] For example, Figure 3 As shown in , four concentric circles can be drawn with the origin of the world coordinate system (0, 0, 0) as the center and the radius r as 10, 20, 30, and 40 respectively. Figure 3 The black solid circle shown can be understood as the center of the sight, or can also be called the center of the sight.

[0093] Then, draw the long scale line. Take the origin (0, 0, 0) as one end point of the line segment and (rcosθ, rsinθ, 0) as the other end point of the line segment to draw a long reference line. Among them, the above r can be 38, and the above θ are -30°, 0°, 30°, 60°, 90°, 120°, 150°, 180° and 210° respectively. Finally, Figure 3 shown.

[0094] Draw short scale lines. Use (r1cosθ, r1sinθ, 0) as one end point of the line segment and (r2cosθ, r2sinθ, 0) as the other end point of the line segment to draw short scale lines. Among them, the above r1 is 38, the above r2 is 42, and the above θ are -30°, -20°, -10°, 0°, 10°, 20°, 30°, 40°, 50°, 60°, 70°, 80°, 90°, 100°, 110°, 120°, 130°, 140°, 150°, 160°, 170°, 180°, 190°, 200° and 210° respectively.

[0095] Finally, you can add angle information next to the long scale line, and the final effect is as follows Figure 3 For ease of understanding, the sight located in the XOY plane may be referred to as the initial sight.

[0096] 8.1.2. Change the sight to the aiming surface

[0097] First, assume that the three-dimensional spatial coordinates of the target point in the puncture path are T (a, b, c), and the three-dimensional spatial coordinates of the needle insertion point are E (m, n, t). Therefore, the normal vector of the aiming plane perpendicular to the puncture path can be obtained: is (ma, nb, tc).

[0098] Secondly, from step 8.1.1, we know that the sight is drawn on the XOY plane, so the normal vector of the initial sight is

[0099] Then, according to the normal vector of the aiming surface and the initial sight normal vector The initial sight is rotated to the aiming surface. This can be achieved by, for example, the following methods:

[0100] The normal vector of the above aiming surface Projection vector projected on the XOY plane (ma, nb, 0), find the projection vector With the X-axis vector n x The angle θ of (1, 0, 0) x , the θ x We can use formula (1) to get:

[0101]

[0102] According to the normal vector of the aiming surface Find the vector With the Z axis vector The angle θ z , the θ z We can use formula (2) to get:

[0103]

[0104] Since the angle range belongs to (0, π), if the aiming surface needs to be rotated to the plane corresponding to the aiming device (i.e., the XOY plane), the first rotation direction is the normal vector of the aiming surface. When the y coordinate nb is greater than 0, the normal vector Rotate the angle θ clockwise around the Z axis with (0, 0, 0) as the rotation point x ; When the y coordinate nb is less than 0, the normal vector Rotate the angle θ counterclockwise around the Z axis with (0, 0, 0) as the rotation point x .

[0105] In the normal vector Rotate θ clockwise or counterclockwise around (0, 0, 0) x Based on this, take (0, 0, 0) as the rotation point and rotate the angle θ clockwise around the Y axis z , at this time, the normal vector The direction is parallel to the Z axis.

[0106] Conversely, if you need to rotate the sight to the aiming surface, first, rotate the sight counterclockwise around the Y axis at an angle of θ with (0, 0, 0) as the rotation point. z Then, when the y coordinate nb is greater than 0, the normal vector Rotate the angle θ counterclockwise around the Z axis with (0, 0, 0) as the rotation point x , when the y coordinate nb is less than 0, the normal vector Rotate the angle θ clockwise around the Z axis with (0, 0, 0) as the rotation point x After the above transformation, the initial aimer originally located in the XOY plane is now located in the aiming plane perpendicular to the puncture path.

[0107] 8.1.3. Move the center of the sight to the puncture path

[0108] In this step, when the puncture path has been determined, the coordinates of any point on the puncture path can be used as the center of the sight. For example, it can be the coordinates of the needle insertion point, or the coordinates of the point halfway between the needle insertion point and the target point.

[0109] In some embodiments, the coordinates of the target point of the puncture path can be used as the center of the sight, and in this case, the sight can be considered to be located at the target sighting surface. For example, since the target coordinates T are (a, b, c), the center of the sight (0, 0, 0) can be translated along the X, Y, and Z axes by distances a, b, and c, respectively, based on step 8.1.2, so that the center of the sight and the target coincide.

[0110] 8.2. Determine the direction of the sight according to the field of view of the optical tracking device

[0111] For ease of understanding, please refer to Figure 4 , Figure 4 It is a schematic diagram of a scenario for determining the direction of a sight according to the field of view direction of an optical tracking device provided in an embodiment of the present application.

[0112] like Figure 4 As shown, the needle insertion point on the puncture path 401 is point E, which can also be called the needle insertion point E; the target point on the puncture path 401 is point T, which can also be called the target point T. The aimer 402 has moved to the target point aiming surface after the above step 8.1, and the center of the aimer is the target point T.

[0113] like Figure 4 As shown in FIG. 1 , the field of view of the optical tracking device 403 is the positive direction of the Z axis, so the direction vector of the field of view of the optical tracking device 403 is (0, 0, 1). Select the unit direction vector in the direction of the sight pointing to the optical tracking device 403 As the intermediate vector, is (0, 0, -1).

[0114] Then, the vector is calculated according to the point to space plane formula Projection vector on the target aiming surface Finally, we get the vector is (x0, y0, z0), where x0, y0, z0 can be expressed by formula (3):

[0115]

[0116] According to the vector angle formula, the 90° direction line, 0° direction line and the above vector are calculated. The angles between them are respectively 90 , angle θ0.

[0117] Finally, the vector The relationship between the 90° direction line and the 0° direction line can be expressed as Figure 5 express, Figure 5 is a vector provided in the embodiment of the present application Schematic diagram of the relationship between the 90° direction line and the 0° direction line.

[0118] Depend on Figure 5 From (a), (b), (c) and (d) in When the angle between the 0° direction line and the puncture path is less than 90°, the aimer rotates clockwise by θ 90 Then the 90° direction line of the sight can be aligned with The direction is consistent. When the angle between the 0° direction line and the puncture path is greater than 90°, the aimer rotates counterclockwise by θ 90 Then the 90° direction line of the sight can be aligned with The direction is consistent.

[0119] It is understandable that when the sight is rotated so that the 90° direction line is aligned with the When the directions are consistent, the puncture path is used as the rotation axis for rotation. This method has a large amount of calculation.

[0120] Therefore, in some embodiments, the angle θ is obtained at 90 After that, the initial sight located in the XOY plane is rotated according to: Rotate around the Z axis with the origin (0, 0, 0) as the rotation point by θ 90 , let the 90° direction of the sight point to the optical tracking device; then rotate the sight to the aiming surface according to the above step 8.1.

[0121] It is understandable that, in addition to aligning the 90° direction line of the sight with the optical tracking device, other direction lines may be aligned with the optical tracking device in a similar manner as described above, such as a 30° direction line or a 180° direction line.

[0122] 8.3. Determine the real-time position of the surgical needle on the aimer

[0123] It is understood that after the above steps 8.1 and 8.2, the aiming device can be positioned at the aiming plane, and the center of the aiming plane is a point on the puncture path, such as the needle insertion point, the target point, etc. The direction of the aiming plane facing the optical tracking device can also be determined, such as the above The sight can be rotated to point in a certain direction towards the optical tracking device.

[0124] In this step, the center of the sight is taken as the target point, and the 90° direction line of the sight points to the optical tracking device. Figure 6a , Figure 6a It is a schematic diagram of a scenario of finding a puncture path based on a sight provided in an embodiment of the present application.

[0125] like Figure 6a As shown in FIG. 6 , the 90° direction line of the sight 605 points to the optical tracking device 603, and the center of the sight 605 is the target point T on the puncture path. Figure 6a The sight 605 shown does not exist in real space. Figure 6a The sights are drawn for ease of understanding.

[0126] similar Figure 1 As described above, the surgical needle 602 is connected to the robot arm 601, and the marker on the surgical needle 602 is within the tracking range of the optical tracking device 603. The optical tracking device 603 and the display device 604 include a communication connection, and the display device 604 can display a virtual puncture path and the surgical needle tracked by the optical tracking device 603. In the embodiment of the present application, the display device 604 can also display a sight 605.

[0127] It is understandable that when the display device 604 displays the sight 605, it can be displayed at any angle of the sight 605. In order to facilitate the doctor's viewing, the sight 605 can be displayed in the field of view of the puncture path. The final display effect is as follows: Figure 6a shown.

[0128] During the operation, the doctor will adjust the position of the surgical needle 602 through the robot arm 601, so the position of the surgical needle 602 changes in real time. The needle body of the surgical needle 602 can be abstracted as a line segment. According to the principle of two points determining a line, the position of the needle body can be determined by real-time tracking of the three-dimensional spatial coordinates of two points (referred to as reference points) on the surgical needle 602 through the optical tracking device 603, for example, the needle tail point and the needle tip point of the surgical needle 602.

[0129] However, when the needle tail point and needle tip point of the surgical needle 602 are used as reference points, the needle tip and the needle tail point will move simultaneously during the fine-tuning process, resulting in a large fine-tuning deviation. Therefore, in order to reduce the fine-tuning deviation, a fixed structure on the robot arm 601 can be used as a reference point, such as a fine-tuning rotation point. In this embodiment, the fine-tuning rotation point is used to support or fix the entire surgical needle, and can be set at one-half or one-third of the needle body of the surgical needle.

[0130] Exemplarily, the fixed surgical needle can be loosened by rotating the fine-tuning rotation point, and after adjusting the angle of the surgical needle, the fine-tuning rotation point can be rotated and tightened to fix the surgical needle. In this embodiment, the coordinates corresponding to the fine-tuning rotation point can be the coordinates of the rotation center point of the structure. It can be understood that when the fine-tuning rotation point is rotated and loosened to adjust the angle of the surgical needle, another reference point on the surgical needle (such as the needle tip or needle tail of the surgical needle) will move, while the fine-tuning rotation point will not move (because the rotation center will not move), which can reduce the difficulty of the operator to align the surgical needle with the puncture path.

[0131] Here, the fine-tuning rotation point and the needle tail point are taken as reference points. For example, when the optical tracking device does not track the surgical needle through the marker, it is assumed that the coordinate system corresponding to the surgical needle is as follows: Figure 6b The coordinate system shown in FIG. 1 is a coordinate system in which the needle tip of the surgical needle is located at the origin of the coordinate system, the initial coordinates of the fine-tuning rotation point for controlling the needle body are (L, 0, 0), and the initial coordinates of the needle tail point are (K, 0, 0). The optical tracking matrix of the optical tracking device 603 is the matrix T, which can be expressed by formula (4):

[0132]

[0133] When the optical tracking device tracks the surgical needle through the marker, the real-time coordinates R of the fine-tuning rotation point and the real-time coordinates E of the needle tail point can be calculated according to the initial coordinates of the fine-tuning rotation point, the initial coordinates of the needle tail point, and the matrix T, where the coordinate R is (R x1 *L+T x , R y1 *L+T y , R z1 *L+T z ), coordinate E is (R x1 *K+T x , R y1 *K+T y , R z1 *K+T z ).

[0134] According to the point-to-space plane formula, the coordinates of the projection point R0 of the fine-tuning rotation point projected onto the plane where the sight 605 is located, and the coordinates of the projection point E0 of the needle tail point onto the plane where the sight 605 is located are calculated, and then the projection points are displayed on the display device 604. It can be understood that when the display device 604 displays the above projection points, the display angle is the same as that of the sight 605. For example, Figure 6a The 606 in the figure can be understood as the fine-tuning rotation point displayed by the display device 604, referred to as the fine-tuning rotation point 606; and the 607 can be understood as the needle tail point displayed by the display device 604, referred to as the needle tail point 607.

[0135] 9. Surgical needle puncture

[0136] According to step 8.3, it can be understood that since the center of the sight is the target point, when the projection point of the reference point (such as the above-mentioned fine-tuning rotation point and the needle tail point) on the sight coincides with the center of the sight, the surgical needle and the puncture path are in the same straight line, and then the surgical needle can be pushed along the straight line to perform the operation.

[0137] Take the above-mentioned fine-tuning rotation point and needle tail point as reference points as an example. In the actual operation, the fine-tuning rotation point can be first made to coincide with the center of the sight, and the needle tail point can be made as close to the fine-tuning rotation point as possible to facilitate fine-tuning. Then, fix the mechanical arm bracket for fine-tuning (the fine-tuning rotation point will not move at this time, but the needle tail point will change position) until the fine-tuning rotation point, the needle tail point and the center point of the sight coincide, and it can be considered that the puncture path alignment is completed.

[0138] For example, see Figure 7 , Figure 7 1 is a schematic diagram of the result of searching for a puncture path based on a sight provided by an embodiment of the present application. Figure 7 As shown, when the three points coincide, the needle body of the surgical needle has coincided with the puncture path.

[0139] The above is an example of a doctor performing percutaneous renal surgery on a patient to introduce the method provided by the present application. Figure 8 The method provided in the embodiment of the present application is introduced as a whole. For example, please refer to Figure 8 , Figure 8 : is a flow chart of a method for displaying the positioning of a surgical needle provided in an embodiment of the present application, the method comprising:

[0140] 801: Obtain a puncture path, where the puncture path is a virtual path for the surgical needle to puncture the target point.

[0141] In the embodiments of the present application, the target point can be understood as a point inside the body of the subject to be punctured by a surgical needle. The subject can be any object that needs to be punctured by a surgical needle. For example, the subject can be a person, such as a patient, a volunteer, etc.; or an animal. The target point can be a lesion point inside the body of the subject, or can also be called a lesion target point, target point, etc.; or it can be a point set artificially in a compliant test experiment (such as an animal experiment).

[0142] In one possible implementation, the positioning display device for the surgical needle can obtain a CT image of an area of ​​interest (such as a lesion area) of the implementation object, and then perform three-dimensional reconstruction of the above-mentioned area of ​​interest, and determine the puncture path according to the position of the target point in the implementation object and the situation around the target point. In another possible implementation, the positioning display device for the surgical needle can include an input device, and the path information input through the input device is used as the above-mentioned puncture path. Exemplarily, the input device can be a keyboard, a mouse, a touch screen, etc.

[0143] It can be understood that the puncture path includes the above-mentioned target point, which can be represented by a line segment in the actual scene, for example Figure 7 The dotted line displayed on the display device can be understood as the planned puncture path. For other descriptions of this step, please refer to the previous article. Figure 1 Related description of step 7.

[0144] 802: Determine a reference plane, wherein an angle between the reference plane and the puncture path is greater than or equal to a first threshold, and the reference plane intersects the puncture path at a first intersection point.

[0145] In this step, the angle between the reference plane and the puncture path can be understood as the angle between the reference plane and the straight line on which the puncture path is located. It can be understood that the angle between the straight line and the plane ranges from 0 to 90°, and the first threshold can be exemplarily any value greater than or equal to 85° and less than or equal to 90°, which can be set according to actual conditions.

[0146] Generally, the closer the angle is to 90°, the better the puncture guidance effect for the operator. Theoretically, when the reference plane is perpendicular to the puncture path, the positioning display effect of the surgical needle is the best, the auxiliary effect on finding the puncture path is the greatest, and the surgical needle can be directly aligned with the puncture path. In this step, when the reference plane is perpendicular to the puncture path, the reference plane can be understood as the aiming plane in the previous embodiment.

[0147] It is understandable that when the above-mentioned angle is not equal to 90°, the operator can perform a first alignment of the puncture path under the guidance of the reference plane, the first projection point and the second projection point, and then fine-tune the surgical needle to perform a second alignment of the puncture path.

[0148] In this step, the first intersection is the intersection between the puncture path and the reference plane. Exemplarily, the first intersection may be the needle insertion point on the puncture path, or other points between the needle insertion point and the target point, such as a point at halfway point, etc. For a detailed introduction, please refer to the description of step 8.1.3 above.

[0149] In some embodiments, the first intersection is the target point. It is understandable that the surgical needle ultimately needs to puncture along the puncture path and pierce the target point, so displaying the target point as the first intersection can better assist doctors and other operators in finding the puncture path.

[0150] In this step, illustratively, the positioning display device of the surgical needle can determine the direction vector corresponding to the puncture path through any two points on the puncture path, and then obtain the reference plane according to the direction vector.

[0151] 803: Display the reference plane and the first intersection point.

[0152] Generally, the puncture path is considered to be fixed after it is planned before a certain operation or experiment and will not change. Although there may be multiple planes with an angle greater than or equal to the first threshold with the puncture path, in the embodiment of the present application, the reference plane should be understood as a fixed plane, for example, the reference plane may be a plane perpendicular to the puncture path, or a plane with an angle of 89° with the puncture path. Therefore, after the puncture path is determined, the reference plane and the first intersection are also determined and will not change during the subsequent operation or experiment.

[0153] For example, Figure 6bAs shown, the reference plane may be the plane where the sight 605 is located, and the first intersection may be the target point T. After the puncture path has been determined, the spatial positions of the reference plane and the first intersection will not change. However, there may be multiple display modes when the positioning display device of the surgical needle displays the reference plane and the first intersection. Exemplarily, it may be displayed at any fixed position on the display screen, such as the upper left, upper right, or lower left corner of the display screen; or, the positioning device of the surgical needle may also receive user operations to change the display position, such as moving from the upper left to the upper right. It is understandable that the movement of the reference plane and the first intersection by the user operation should be understood as the movement of the display position, which is a change in the display effect, while the spatial position of the reference plane and the first intersection has not changed.

[0154] For example, Figure 6a As shown in FIG. 6 , the aiming device 605 and the target T may be displayed at the upper right of the display device 604, or at the lower right of the display device 604, or may be moved to the middle of the display device 604 in response to a user operation. It is understood that no matter how the display position is changed, the aiming device 605 and the target T are in the real space (e.g., Figure 6a The position of the scene shown) remains unchanged.

[0155] In this step, displaying the reference plane can be understood as displaying a portion of the reference plane, such as a certain area including the first intersection, and should not be understood as displaying a plane in a mathematical sense. Exemplarily, the positioning display device for the surgical needle can display a circular area on the reference plane with the first intersection as the center, or a square area, a rectangular area or other irregular area with the first intersection as the center.

[0156] In this step, auxiliary lines may be included when displaying the reference plane. Figure 8 The method shown also includes: displaying second auxiliary lines, wherein the second auxiliary lines include at least one concentric circle auxiliary line with the first intersection as the center, and a plurality of angle auxiliary lines on the concentric circle auxiliary lines.

[0157] By displaying the second auxiliary line, the present embodiment can better assist the operator in controlling the position of the surgical needle, and further reduce the time spent on the alignment process of aligning the surgical needle with the puncture path. For example, the reference plane including the second auxiliary line in the present embodiment can be understood as Figure 3 Sights shown.

[0158] 804: Determine a first projection point of the first reference point on the reference plane, and determine a second projection point of the second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle.

[0159] In an embodiment of the present application, the first reference point and the second reference point are determined according to the structure of the surgical needle. A line is determined based on two non-overlapping points, and the position of the surgical needle can be determined by two reference points on the surgical needle (i.e., the first reference point and the second reference point). Exemplarily, the first reference point is the tail of the surgical needle, and the second reference point is the tip of the surgical needle; or, the first reference point is the tail of the surgical needle, and the second reference point is a rotation point for controlling the surgical needle. Among them, the rotation point can be understood as the fine-tuning rotation point in the previous embodiment, and please refer to the description of step 803 above for details.

[0160] In this step, the surgical needle is clamped by a clamp, and at least three markers are fixedly set on the clamp. The spatial coordinates of the first reference point and the second reference point are determined by collecting the coordinates of the at least three markers by an optical tracking device. For details, please refer to the above Figure 1 and Figure 2 That is to say, the coordinates of the first reference point and the second reference point can be determined by tracking the marker with an optical tracking device and the relative position relationship between the first reference point, the second reference point and the marker, and then the first projection point and the second projection point can be calculated according to the point-to-space plane formula. For details, please refer to the description of step 8.3 above.

[0161] 805: Display the first projection point and the second projection point.

[0162] It is understandable that the surgical needle actually exists in the real space, and the surgical needle needs to be constantly moved to align with the puncture path. Therefore, the position of the surgical needle changes, and the above-mentioned first projection point and the above-mentioned second projection point also change.

[0163] Likewise Figure 6a As an example, the reference plane may be the plane where the aiming device 605 is located, and the first intersection point may be the target point T. When the surgical needle is not within the tracking range of the optical tracking device 603, the display device 604 will not display the projection point 606 and the projection point 607. When the surgical needle is within the tracking range of the optical tracking device 603, the display device 604 will display the projection point of the reference point tracked by the optical tracking device 603.

[0164] In the embodiment of the present application, the positioning display device of the surgical needle can distinguish and display the first intersection point, the first projection point and the second projection point, for example, by different colors, different sizes and different shapes, and can also mark each point.

[0165] The method provided in the embodiment of the present application obtains the virtual puncture path of the surgical needle, determines a reference plane whose angle with the puncture path is greater than or equal to a first threshold value according to the puncture path, displays the reference plane and the first intersection point between the reference plane and the puncture path; when displaying the surgical needle, displays the first projection point of the first reference point on the reference plane, and displays the second projection point of the second reference point on the reference plane. Since the ultimate goal of the surgical needle is to align with the puncture path, the present application can allow the operator to know the position of the surgical needle more clearly and intuitively, such as the inclination angle of the surgical needle, by displaying the reference plane, the first intersection point and the projection point of the reference point on the reference plane. If the distance between the first projection point and the second projection point is too far, it can be considered that the inclination angle of the surgical needle is too large; it can also allow the operator to know the azimuth distance between the surgical needle and the puncture path more clearly and intuitively, such as the surgical needle is on the right side of the puncture path, etc. Therefore, the above method can allow the operator to have a sense of direction when moving the surgical needle. The operator uses the first intersection point as the target point and makes the first projection point and the second projection point coincide with the first intersection point as much as possible, which means that the surgical needle and the puncture path are roughly aligned. The position of the surgical needle can be fine-tuned subsequently to complete the alignment of the puncture path, greatly reducing the user's operating difficulty and further reducing the time spent on the alignment process of aligning the surgical needle with the puncture path.

[0166] In particular, when the puncture path is perpendicular to the reference plane, the surgical needle can be moved so that the first projection point, the second projection point and the first intersection point coincide with each other, and the surgical needle can be considered to be aligned with the puncture path.

[0167] In addition to displaying the reference plane and the first intersection point, auxiliary lines may also be displayed according to the reference direction. Figure 8 The above method, before step 804, further includes:

[0168] 806: Determine a projection direction vector of a reference direction vector on the reference plane, where the reference direction vector is a fixed direction determined based on the real space of the surgical needle.

[0169] 807: Display the first auxiliary line corresponding to the projection direction vector.

[0170] In the embodiment of the present application, the direction corresponding to the reference direction vector can be called the reference direction, which is a fixed direction determined by the real space of the surgical needle, wherein the real space of the surgical needle can be understood as the real space where the surgical needle is used, such as the operating room space, the experimental space of the animal experiment. The above reference direction is fixed in the process of aligning the surgical needle with the puncture path. By displaying the reference direction in the real space, the operator can have a better sense of direction in the process of moving the surgical needle and more easily grasp the moving direction of the surgical needle, thereby reducing the time spent on the alignment process of aligning the surgical needle with the puncture path and improving the puncture efficiency.

[0171] In the embodiment of the present application, the above-mentioned reference direction can be the direction in which the target point points to the optical tracking device (referred to as the first direction), the direction in which the target point points to the feet of the implementation object (referred to as the second direction), and the direction in which the target point points to the head of the implementation object (referred to as the third direction).

[0172] It is understandable that, generally during the operation, the subject will not move, for example, if anesthetized in advance; the optical tracking device will not move either, so the first direction, the second direction and the third direction can all be considered fixed. If the reference direction is the second direction, the operator can move the surgical needle with reference to the subject's feet, and in order to achieve a better tracking effect of the surgical needle, the optical tracking device can be placed near the feet; if the reference direction is the first direction or the third direction, it is similar.

[0173] In this embodiment, there are many ways to determine the direction vector of the reference direction, such as with the help of a surgical needle. For example, the surgical needle is placed parallel to the object, and after the optical tracking device tracks the marker on the surgical needle, the coordinates of the needle tip and the needle tail can be determined through the optical tracking matrix (such as formula (4)), and the second direction or the third direction can be determined based on the coordinates of the needle tip and the needle tail. It should be understood that other reference directions can be set in the same way.

[0174] It is understandable that, in order to achieve a better tracking effect for the surgical needle, the surgical needle will be facing the optical tracking device. Therefore, the error of determining the direction vector of the third direction is minimized, and the effect and user experience of the reference direction being the first direction are better. When the reference direction is the first direction, the specific determination process can refer to the description of step 802 above.

[0175] It is understandable that after the reference direction vector is determined, the projection direction vector can be calculated according to the point-to-space plane formula. It should also be understood that the reference plane is determined, the reference direction is also determined, and therefore, the projection direction vector and the corresponding first auxiliary line are also determined, that is, the display effect of the first auxiliary line can be changed, but the spatial position of the projection vector corresponding to the first auxiliary line is not changed.

[0176] In this embodiment, the display order of the above step 807 and the above step 803 is not limited. The above reference plane, the above first intersection and the above first auxiliary line can be displayed together, or the above reference plane and the above first intersection can be displayed first, and then the above first auxiliary line can be displayed. When displaying the above first auxiliary line, line segments can be displayed, and specific information of the reference direction can be marked. Other methods (such as color, thickness, shape, etc.) can also be used to distinguish it from other auxiliary lines (such as the second auxiliary line) in other embodiments of the present application. This application does not limit this.

[0177] Optionally, the reference direction may be one or more than one. For example, the auxiliary line corresponding to the first direction may be displayed, or the auxiliary lines corresponding to the first direction and the second direction may be displayed together.

[0178] It is understandable that, in order to simplify the calculation steps, the positioning display device of the surgical needle can first create an initial point and an initial auxiliary line on the coordinate plane, and then rotate and translate the coordinate plane to transform the coordinate plane into a reference plane, convert the initial point into the first intersection point, and convert the initial auxiliary line into the first auxiliary line and / or the second auxiliary line. The coordinate plane can be an XOY plane, an XOZ plane, and a YOZ plane.

[0179] Therefore, in some embodiments, the determining of the reference plane, displaying the reference plane and the first intersection point, and displaying the first auxiliary line corresponding to the projection direction vector include:

[0180] (a) taking the coordinate plane of the spatial coordinate system where the puncture path is located as a first plane, wherein the first plane includes a first candidate point and a first candidate line;

[0181] In this embodiment, the spatial coordinate system where the puncture path is located can be understood as the coordinate system corresponding to the optical tracking device, that is, the world coordinate system. Exemplarily, the XOY plane can be used as the above-mentioned first plane, and a point in XOY can be used as the above-mentioned first candidate point, such as the origin. The above-mentioned first candidate line can be understood as a line segment, for example, it can be a line segment with the above-mentioned first intersection as an endpoint, or the straight line where the above-mentioned first candidate line is located includes the above-mentioned first intersection.

[0182] (b) determining an orientation transformation matrix and a rotation angle, wherein the orientation transformation matrix is ​​used to rotate the first plane to a position where the angle between the first plane and the puncture path is greater than or equal to the first threshold value and the first candidate point is located on the puncture path; the rotation angle is the angle between the direction vector of the first candidate line and the projection direction vector;

[0183] In this step, the above-mentioned orientation transformation matrix may include a rotation component and a translation component. For example, as described in the previous step 8.1.2, the rotation component can be determined based on the angle between the normal vector of the first plane and the vector corresponding to the puncture path, and the translation component can be obtained based on the distance between the above-mentioned first candidate point and the second candidate point on the puncture path, thereby obtaining the above-mentioned orientation transformation matrix.

[0184] This application does not make any specific restrictions on the specific values ​​of the above-mentioned rotation component and translation component. For example, for the rotation component, as long as the above-mentioned first plane is transformed based on the above-mentioned orientation transformation matrix, the first plane can be rotated to an angle greater than or equal to the above-mentioned first threshold value with respect to the above-mentioned puncture path. For example, based on the rotation component, the first plane can be rotated to a position perpendicular to the above-mentioned puncture path, or the first plane can be rotated to a position with an angle of 89° with respect to the above-mentioned puncture path. It can be understood that different angles correspond to different rotation components. For example, for the translation component, as long as the above-mentioned first plane is transformed based on the above-mentioned orientation transformation matrix, the position of the first candidate point on the above-mentioned puncture path can be made. For example, the first candidate point can be transformed to a target point on the puncture path, or it can be transformed to other points on the puncture path. It can be understood that different translation components correspond to different points on the puncture path.

[0185] In this step, after determining the above-mentioned orientation transformation matrix, the above-mentioned first plane can be transformed based on the above-mentioned orientation transformation matrix to obtain the transformed first plane, and then the above-mentioned rotation angle can be determined. The transformed first plane includes the transformed first candidate line, and the above-mentioned transformed first candidate line is obtained based on the rotation and translation of the above-mentioned first candidate line, similar to step 8.1.2 in the above-mentioned embodiment, the sight of the XOY plane is transformed to the aiming plane.

[0186] In this step, the angle between the projection direction vector of the reference direction vector on the transformed first plane and the direction vector of the transformed first candidate line can be calculated to obtain the rotation angle. Exemplarily, when the first plane is a sight, the first candidate line is a 90° line in the sight, and the reference direction is the first direction (i.e., the direction pointing to the optical tracking device), the details of this step can also refer to the description of step 8.2 above.

[0187] (c) In the coordinate plane, the first plane is rotated based on the rotation angle to obtain a second plane.

[0188] It can be understood that after the above-mentioned orientation transformation matrix and the above-mentioned rotation angle have been determined, the above-mentioned first plane can be transformed to the aiming plane based on the above-mentioned orientation transformation matrix to obtain the transformed first plane; then, the transformed first plane is rotated based on the above-mentioned rotation angle with the puncture path as the axis.

[0189] However, since the puncture path is a path in space, there is often a certain angle between it and each coordinate axis, and the amount of calculation required to rotate with the puncture path as the axis is relatively large. Therefore, in this embodiment, the first plane is first rotated in the coordinate plane. It can be understood that the rotation axis used when rotating the coordinate plane is the coordinate axis, and the amount of calculation is small. Exemplarily, when the above coordinate plane is the XOY plane, the rotation is performed with the Z axis as the reference.

[0190] (d) transforming the second plane based on the orientation transformation matrix to obtain the reference plane, the first intersection point and the first auxiliary line; the first auxiliary line is obtained by rotating and translating the first candidate line; and the first intersection point is obtained by translating the first candidate point;

[0191] It can be understood that the above-mentioned reference plane can be obtained by transforming the above-mentioned second plane based on the above-mentioned orientation transformation matrix, wherein the first auxiliary line in the reference plane can be understood as the first auxiliary line obtained by rotation and translation of the above-mentioned rotation angle and the above-mentioned orientation transformation matrix, and the first intersection point in the reference plane can be understood as the first line obtained by translation of the translation component in the above-mentioned orientation transformation matrix.

[0192] (e) Display the reference plane, the first intersection point and the first auxiliary line.

[0193] It can be understood that the first plane may include not only the first candidate line and the first candidate point, but also a second candidate line. The second candidate line is first rotated by the rotation angle and then transformed by the orientation transformation matrix to obtain the second auxiliary line in the embodiment of the present application. For details, please refer to the description of steps 8.1, 8.2 and 8.3 above.

[0194] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

[0195] See also Fig. 9 , Fig. 9is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 90 is used to execute the positioning display method of the surgical needle provided in the above embodiment. It can be understood that any device that can implement the method provided in the present application belongs to the protection scope of the present application. Exemplarily, the electronic device 90 can be a laptop computer or a desktop computer, etc., which is not limited in the embodiment of the present application.

[0196] like Fig. 9 As shown, the electronic device 90 includes an acquisition unit 901, a determination unit 902, and a display unit 903, and optionally, may also include a transformation unit 904. The description of each unit is as follows:

[0197] An acquisition unit 901 is used to acquire a puncture path, where the puncture path is a virtual path taken by the surgical needle toward the target point;

[0198] A determination unit 902 is used to determine a reference plane, wherein the angle between the reference plane and the puncture path is greater than or equal to a first threshold, and the reference plane intersects the puncture path at a first intersection point;

[0199] A display unit 903, configured to display the reference plane and the first intersection point;

[0200] The determining unit 902 is further configured to determine a first projection point of the first reference point on the reference plane, and to determine a second projection point of the second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle;

[0201] The display unit 903 is further configured to display the first projection point and the second projection point.

[0202] Optionally, the determining unit 902 is further configured to determine a projection direction vector of a reference direction vector on the reference plane, wherein the reference direction vector is a fixed direction determined according to the real space of the surgical needle;

[0203] The display unit 903 is further configured to display a first auxiliary line corresponding to the projection direction vector.

[0204] Optionally, the determining unit 902 is specifically configured to use a coordinate plane of the space coordinate system where the puncture path is located as a first plane, where the first plane includes a first candidate point and a first candidate line;

[0205] The determination unit 902 is specifically used to determine an orientation transformation matrix and a rotation angle, wherein the orientation transformation matrix is ​​used to rotate the first plane to a position where the angle between the first plane and the puncture path is greater than or equal to the first threshold value and the first candidate point is on the puncture path; the rotation angle is the angle between the direction vector of the direction of the first candidate line and the projection direction vector;

[0206] The above device further includes a transformation unit 904, configured to rotate the first plane based on the rotation angle in the coordinate plane to obtain a second plane;

[0207] The transformation unit 904 is further configured to transform the second plane based on the orientation transformation matrix to obtain the reference plane, the first intersection point and the first auxiliary line; the first auxiliary line is obtained by rotating and translating the first candidate line; the first intersection point is obtained by translating the first candidate point;

[0208] The display unit 903 is further configured to display the reference plane, the first intersection point, and the first auxiliary line.

[0209] Optionally, the surgical needle is clamped by a holder, at least three markers are fixedly arranged on the holder, and the spatial coordinates of the first reference point and the second reference point are determined by collecting the coordinates of the at least three markers by an optical tracking device; the reference direction vector includes any one or more of the following:

[0210] The target point includes a direction vector pointing to the direction of the optical tracking device, a direction vector pointing to the direction of the feet of the subject, and a direction vector pointing to the direction of the head of the subject.

[0211] Optionally, the first intersection point is the target point.

[0212] Optionally, the display unit 903 is further configured to display second auxiliary lines, where the second auxiliary lines include at least one concentric circle auxiliary line with the first intersection as the center, and a plurality of angle auxiliary lines on the concentric circle auxiliary lines.

[0213] See also Fig.10 , Fig.10 1 is a schematic diagram of the structure of another electronic device provided in an embodiment of the present application. The electronic device 100 can be used to implement the above-mentioned surgical needle positioning display method.

[0214] like Fig.10 As shown, the electronic device 100 includes a memory 1001 and a processor 1002. Optionally, the electronic device 100 may also include a communication interface 1003 and a bus 1004; further optionally, the electronic device 100 may also include a display screen 1005. The memory 1001, the processor 1002, the communication interface 1003 and the display screen 1005 are connected to each other through the bus 1004.

[0215] The memory 1001 is used to provide a storage space, in which data such as an operating system and a computer program can be stored. The memory 1001 includes, but is not limited to, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a portable read-only memory (CD-ROM).

[0216] The processor 1002 is a module for performing arithmetic operations and logical operations, and may be a combination of one or more of processing modules such as a central processing unit (CPU), a graphics processing unit (GPU), or a microprocessor unit (MPU). In addition, a computer program is stored in the memory 1001, and the processor 1002 may call the computer program stored in the memory 1001 to execute the corresponding method.

[0217] The display screen 1005 is used to implement the display function of the electronic device 100. Exemplarily, the display screen 1005 can be used to display the aimer, the first projection point, the second projection point, and can also be used to display the puncture path, etc. In some embodiments, the display screen 1005 can implement the functions of the display device 106 and the display device 604 in the above embodiments.

[0218] In the present application embodiment, Fig.10 When the electronic device 100 shown executes the above-mentioned surgical needle positioning display method, the processor 1002 can control the display function of the display screen 1005 and can also control the data communication function of the communication interface 1003.

[0219] Exemplarily, in some embodiments, the processor 1002 is used to obtain a puncture path;

[0220] Processor 1002, configured to determine a reference plane, determine a first projection point of a first reference point on the reference plane, and determine a second projection point of a second reference point on the reference plane;

[0221] The processor 1002 is used to control the display screen 1005 to display the reference plane, the first intersection point, the first projection point and the second projection point.

[0222] In another embodiment, the processor 1002 may be used to implement the functions of the acquisition unit 901, the determination unit 902, and the transformation unit 904 in the electronic device 90; the display screen 1005 may be controlled by the processor 1002 to implement the function of the display unit 903 in the electronic device 90. Optionally, the data acquired by the acquisition unit 901 in the electronic device 90 may also be acquired through the communication interface 1003.

[0223] The embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a processor, the method in the embodiment of the present application can be implemented.

[0224] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the above claims.

Claims

1. A method for positioning and displaying a surgical needle, characterized in that: The method comprises: Acquire a puncture path, where the puncture path is a virtual path of a surgical needle puncturing a target point; Determine a reference plane, wherein the angle between the reference plane and the puncture path is greater than or equal to a first threshold, and the reference plane intersects the puncture path at a first intersection point; displaying the reference plane and the first intersection point; Determine a first projection point of a first reference point on the reference plane, and determine a second projection point of a second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle; Display the first projection point and the second projection point; The determining of the reference plane comprises: Taking the coordinate plane of the space coordinate system where the puncture path is located as the first plane, the first plane includes the first candidate point and the first candidate line; Determine an orientation transformation matrix and a rotation angle, wherein the orientation transformation matrix is ​​used to rotate the first plane to a position where the angle between the first plane and the puncture path is greater than or equal to the first threshold and the first candidate point is on the puncture path; the rotation angle is the angle between the direction vector of the direction of the first candidate line and the projection direction vector; In the coordinate plane, rotating the first plane based on the rotation angle to obtain a second plane; The second plane is transformed based on the orientation transformation matrix to obtain the reference plane.

2. The method according to claim 1, characterized in that Before determining a first projection point of the first reference point on the reference plane and determining a second projection point of the second reference point on the reference plane, the method further comprises: Determine a projection direction vector of a reference direction vector on the reference plane, wherein the reference direction vector is a fixed direction determined according to the real space of the surgical needle; A first auxiliary line corresponding to the projection direction vector is displayed.

3. The method according to claim 2, characterized in that The method further comprises: The second plane is transformed based on the orientation transformation matrix to obtain the first intersection point and the first auxiliary line; the first auxiliary line is obtained by rotating and translating the first candidate line; and the first intersection point is obtained by translating the first candidate point.

4. The method according to claim 2 or 3, characterized in that: The surgical needle is clamped by a holder, and at least three markers are fixedly arranged on the holder. The spatial coordinates of the first reference point and the second reference point are determined by collecting the coordinates of the at least three markers by an optical tracking device; the reference direction vector includes any one or more of the following: The target point includes a direction vector pointing to the direction of the optical tracking device, a direction vector pointing to the direction of the feet of the subject, and a direction vector pointing to the direction of the head of the subject.

5. The method according to any one of claims 1 to 3, characterized in that: The first intersection point is the target point.

6. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Displaying second auxiliary lines, wherein the second auxiliary lines include at least one concentric circle auxiliary line with the first intersection as the center, and a plurality of angle auxiliary lines on the concentric circle auxiliary line.

7. The method according to any one of claims 1 to 3, characterized in that The first reference point is the needle tail of the surgical needle, and the second reference point is the needle tip of the surgical needle, or is a rotation point used to control the surgical needle.

8. A surgical needle positioning display device, characterized in that: The device comprises: An acquisition unit, used for acquiring a puncture path, wherein the puncture path is a virtual path of a surgical needle puncturing a target point; a determination unit, configured to determine a reference plane, wherein an angle between the reference plane and the puncture path is greater than or equal to a first threshold value, and the reference plane intersects the puncture path at a first intersection point; A display unit, configured to display the reference plane and the first intersection point; The determination unit is further used to determine a first projection point of the first reference point on the reference plane, and to determine a second projection point of the second reference point on the reference plane; the first reference point and the second reference point are determined according to the structure of the surgical needle; The display unit is further used to display the first projection point and the second projection point; The determining unit is specifically configured to use a coordinate plane of the space coordinate system where the puncture path is located as a first plane, wherein the first plane includes a first candidate point and a first candidate line; The determination unit is specifically used to determine an orientation transformation matrix and a rotation angle, wherein the orientation transformation matrix is ​​used to rotate the first plane to a position where the angle between the first plane and the puncture path is greater than or equal to the first threshold value and the first candidate point is on the puncture path; the rotation angle is the angle between the direction vector of the direction of the first candidate line and the projection direction vector; A transformation unit, configured to rotate the first plane based on the rotation angle in the coordinate plane to obtain a second plane; The transformation unit is further used to transform the second plane based on the orientation transformation matrix to obtain the reference plane.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a display screen, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions so that the method according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the method according to any one of claims 1 to 7 is executed.