Method for determining coordinates and related device

Abstract

The embodiment of the present application provides a coordinate determination method and related device, wherein the method comprises the following steps: determining a reference plane, the reference plane is obtained according to the first coordinates of every three markers in N markers at a reference moment, the first coordinates are collected by an optical measuring device, and N is an integer greater than or equal to 3; obtaining M second coordinates of the N markers at a first moment, the second coordinates are collected by the optical measuring device, M is an integer greater than or equal to 3, and the first moment is later than the reference moment; determining at least three coordinates for registration from the M second coordinates, the included angle between a first plane and a second plane is less than or equal to a first threshold value, the first plane is a plane obtained according to three coordinates in the at least three coordinates for registration, and the second plane is one of the reference planes. The method provided by the present application can save the time spent in the registration process and improve the registration efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to methods and devices for determining coordinates. Background Technology

[0002] With the continuous development of computer science and technology, medical technology has also ushered in major breakthroughs, leading to the increasingly widespread application of minimally invasive surgery. Minimally invasive surgery relies on surgical navigation systems, which can be understood as accurately mapping the patient's preoperative or intraoperative computed tomography (CT) image data (i.e., CT images) to the patient's real-time anatomical structures on the operating table. During the surgeon's operation, the system tracks the surgical probe (also called the surgical needle) and updates and displays the position of the surgical needle on the patient's image in real time as a virtual probe.

[0003] In optical tracking-based surgical navigation systems, the coordinates of the surgical needle are obtained based on the coordinate system corresponding to the optical measurement equipment, while the CT image is displayed based on the coordinate system corresponding to the CT equipment. Because the two coordinate systems are different, misalignment will occur when they are displayed on the same interface. Currently, markers can be attached to the patient's body surface. The coordinates of these markers determined by the CT equipment are used as point cloud A, and the coordinates of the markers determined by the optical measurement equipment are used as point cloud B. A point cloud registration algorithm is used to register point clouds A and B. The resulting registration matrix transforms the surgical object in the CT coordinate system to the optical measurement coordinate system for display, achieving coordinate system unification and thus enabling surgical navigation.

[0004] However, in actual surgical environments, flashing points are often unavoidable. Therefore, the detection results of optical measurement equipment may include the coordinates of the markers pasted on the surgical patient's body surface as well as the coordinates of the aforementioned flashing points, resulting in a long registration process and low registration efficiency. Summary of the Invention

[0005] This application provides a method and related apparatus for determining coordinates. The method provided by this application can save time spent in the registration process and improve registration efficiency.

[0006] In a first aspect, embodiments of this application provide a method for determining coordinates, including:

[0007] A reference plane is determined. The reference plane is obtained based on the first coordinates of every three markers out of N markers at a reference time. The first coordinates are acquired by an optical measurement device. N is an integer greater than or equal to 3.

[0008] Obtain M second coordinates of the above N markers at the first time point. The second coordinates are acquired by the above optical measurement device. M is an integer greater than or equal to 3. The first time point is later than the reference time point.

[0009] When M is greater than N, at least three coordinates for registration are determined from the M second coordinates. The angle between the first plane and the second plane is less than or equal to a first threshold. The first plane is a plane obtained from three of the at least three coordinates for registration. The second plane is one of the reference planes.

[0010] In one possible implementation, the above-described determination of the reference plane includes:

[0011] In response to the first triggering operation, the reference plane is determined. The first triggering operation is used to indicate that the user implements the object in a first state. The surface of the implements object is affixed with the N markers.

[0012] The above-mentioned acquisition of the M second coordinates of the N markers at the first moment includes:

[0013] In response to the second triggering operation, the M second coordinates of the N markers at the first time are obtained. The second triggering operation is used to indicate that the implementation object is in a second state, and the stability of the first state is greater than the stability of the second state.

[0014] In one possible implementation, the absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to a second threshold, wherein the first figure is a triangle formed by three coordinates corresponding to the first plane, and the second figure is a triangle formed by three coordinates corresponding to the second plane.

[0015] Alternatively, the absolute value of the difference between the area of ​​the first graphic and the area of ​​the second graphic is less than or equal to the third threshold.

[0016] In one possible implementation, determining at least three coordinates for registration from the M second coordinates includes:

[0017] Three coordinates for registration are determined from the above M second coordinates, and the angle between the plane corresponding to the above three coordinates and the above second plane is less than or equal to the above first threshold.

[0018] The remaining coordinates are combined with two of the three coordinates used for registration to form a third plane. The remaining coordinates are any coordinates among the M second coordinates other than the three coordinates used for registration.

[0019] If the angle between the third plane and the second plane is less than or equal to the first threshold, the remaining coordinates are used as the coordinates for registration.

[0020] In one possible implementation, using the coordinates corresponding to the third plane as coordinates for registration includes:

[0021] If the absolute value of the difference between the perimeter of the third figure and the perimeter of the second figure is less than or equal to the second threshold, the coordinates corresponding to the third plane are used as coordinates for registration, and the third figure is a triangle formed by the three coordinates corresponding to the third plane.

[0022] Alternatively, if the absolute value of the difference between the area of ​​the third graphic and the area of ​​the second graphic is less than or equal to the third threshold, the coordinates corresponding to the third plane are used as the coordinates for registration.

[0023] In one possible implementation, the at least three coordinates used for registration are used to register with a third coordinate, which is obtained by segmenting the N markers in the computed tomography (CT) image, which is acquired by the CT device at the reference time.

[0024] Secondly, embodiments of this application provide a coordinate determining device, comprising:

[0025] The determining unit is used to determine the reference plane, which is obtained based on the first coordinates of every three markers out of N markers at a reference time. The first coordinates are acquired by an optical measuring device, and N is an integer greater than or equal to 3.

[0026] The acquisition unit is used to acquire M second coordinates of the N markers at a first moment, wherein the second coordinates are acquired by the optical measurement device, M is an integer greater than or equal to 3, and the first moment is later than the reference moment.

[0027] The aforementioned determining unit is further configured to determine at least three coordinates for registration from the aforementioned M second coordinates, wherein the angle between the first plane and the second plane is less than or equal to a first threshold, the aforementioned first plane is a plane obtained based on three of the aforementioned at least three coordinates for registration, and the aforementioned second plane is one of the aforementioned reference planes.

[0028] In one possible implementation, the above-mentioned device further includes a response unit for responding to a first trigger operation and controlling the determination unit to determine the reference plane. The first trigger operation is used to indicate that the user implements the object in a first state, and the surface of the object is affixed with the above-mentioned N markers.

[0029] The aforementioned response unit is also used to respond to the second trigger operation and control the aforementioned acquisition unit to acquire the aforementioned M second coordinates of the aforementioned N markers at the aforementioned first time. The aforementioned second trigger operation is used to indicate that the aforementioned implementation object is in a second state, and the stability level corresponding to the aforementioned first state is greater than the stability level corresponding to the aforementioned second state.

[0030] In one possible implementation, the absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to a second threshold, wherein the first figure is a triangle formed by three coordinates corresponding to the first plane, and the second figure is a triangle formed by three coordinates corresponding to the second plane.

[0031] Alternatively, the absolute value of the difference between the area of ​​the first graphic and the area of ​​the second graphic is less than or equal to the third threshold.

[0032] In one possible implementation, the determining unit is specifically used to determine three coordinates for registration from the M second coordinates, wherein the angle between the plane corresponding to the three coordinates for registration and the second plane is less than or equal to the first threshold.

[0033] The aforementioned device further includes a component unit for forming a third plane by combining the remaining coordinates with two of the three coordinates used for registration, wherein the remaining coordinates are any coordinates other than the three coordinates used for registration among the M second coordinates.

[0034] The aforementioned determining unit is specifically used to use the remaining coordinates as coordinates for registration when the angle between the third plane and the second plane is less than or equal to the first threshold.

[0035] In one possible implementation, the determining unit is specifically used to use the coordinates corresponding to the third plane as coordinates for registration when the absolute value of the difference between the perimeter of the third figure and the perimeter of the second figure is less than or equal to the second threshold. The third figure is a triangle formed by the three coordinates corresponding to the third plane.

[0036] Alternatively, the aforementioned determining unit is specifically used to use the coordinates corresponding to the third plane as coordinates for registration when the absolute value of the difference between the area of ​​the third graphic and the area of ​​the second graphic is less than or equal to the third threshold.

[0037] In one possible implementation, the at least three coordinates used for registration are used to register with a third coordinate, which is obtained by segmenting the N markers in the computed tomography (CT) image, which is acquired by the CT device at the reference time.

[0038] Thirdly, embodiments of this application disclose an electronic device, including: a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program stored in the memory to perform a method as described in the first aspect or any possible implementation of the first aspect.

[0039] Fourthly, this application also provides another electronic device, comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs a method as described in the first aspect or any possible implementation thereof.

[0040] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on one or more processors, causes the method as described in the first aspect or any possible implementation thereof to be executed.

[0041] In a sixth aspect, embodiments of this application provide a computer program product including program instructions that, when executed by a processor, cause the processor to perform a method as described in the first aspect or any possible implementation thereof. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments or background art of this application, the accompanying drawings used in the embodiments or background art of this application will be briefly introduced below.

[0043] Figure 1 This is a schematic diagram of an optical measurement system 100 provided in an embodiment of this application;

[0044] Figure 2 This is a schematic diagram of a scenario provided by an embodiment of this application for registering the coordinate system corresponding to an optical measurement device with the coordinate system corresponding to a CT device;

[0045] Figure 3 This is a schematic diagram illustrating a scenario where an optical measuring device generates artifact points during the acquisition of the coordinates of a marker, as provided in an embodiment of this application.

[0046] Figure 4 This is a flowchart illustrating a method for determining coordinates provided in an embodiment of this application;

[0047] Figure 5 This is a flowchart illustrating another method for determining coordinates provided in an embodiment of this application;

[0048] Figure 6 This is a flowchart illustrating yet another method for determining coordinates provided in this application.

[0049] Figure 7 This is a flowchart illustrating a registration method provided in an embodiment of this application;

[0050] Figure 8 This is a schematic diagram illustrating the registration of the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device at time B, according to an embodiment of this application.

[0051] Figure 9 This is a schematic diagram of the structure of a coordinate determining device provided in an embodiment of this application;

[0052] Figure 10 This is a schematic diagram of another coordinate determining device provided in an embodiment of this application. Detailed Implementation

[0053] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items. The terms “first” and “second,” etc., in the specification, claims, and drawings of this application are used to distinguish different objects and not to describe a particular order.

[0054] With the continuous development of computer science and technology, medical technology has also achieved significant breakthroughs, leading to the increasingly widespread application of minimally invasive surgery. Compared to open surgery, minimally invasive surgery is characterized by smaller incision areas, lower infection rates, faster patient recovery, and shorter hospital stays. However, because of the smaller incisions, surgeons cannot directly visualize the patient's internal structures, resulting in a loss of direct visual feedback. Furthermore, the limited intraoperative field of view necessitates repeated review of preoperative images, making minimally invasive surgery reliant on surgical navigation systems.

[0055] In this embodiment, the surgical navigation system can be understood as accurately mapping the patient's preoperative CT image data (i.e., CT images) to the patient's real-time anatomical structures on the operating table. During the surgery, the system tracks the surgical probe (also called the surgical needle) and updates its position on the patient's images in real time as a virtual probe. The surgical navigation system allows doctors to determine the position of the surgical needle relative to the patient's anatomical structures, making minimally invasive surgery faster, more precise, and safer. For example, in brain surgery, the surgical navigation system can display a "virtual human brain" of the patient on a monitor. During the surgery, the system displays the real-time position of the surgical needle in the doctor's hand, allowing the doctor to know whether the tumor edge has been reached, whether there is important tissue in front of the probe, etc., assisting the doctor in the surgery.

[0056] In this embodiment, an optical measurement device can be used to map surgical needles, the patient's body, and other objects in a real-world scenario onto a display screen in real time. This allows doctors to obtain more information through the display, thereby improving surgical precision and reducing the doctor's workload. For example, please refer to... Figure 1 , Figure 1 This is a schematic diagram of an optical measurement system 100 provided in an embodiment of this application. Figure 1 As shown, the optical measurement system 100 includes an optical measurement device 101, an electronic device 102, and a marker 103. Additionally, Figure 1 The part 104 in the document can be understood as the implementation object, and for ease of understanding, it will be referred to as implementation object 104 thereafter. In the implementation of this application, the implementation object can be understood as a patient (e.g., a patient who needs to undergo surgery), or, in some scenarios, it can also be understood as other objects or users that need to collect data through the aforementioned optical measurement system. For ease of understanding, the term "implementation object" will be used as the main subject in the following explanation. Figure 1 The part 105 in the document can be understood as the communication connection between the optical measuring device 101 and the electronic device 102. For ease of understanding, it will be referred to as communication connection 105. It should be understood that the above-mentioned communication connection 105 can be a wired connection or a wireless connection, and this application does not limit it in this regard.

[0057] In this embodiment, the marker 103 can be either a passive or an active marker. A passive marker has a reflective coating on its surface that reflects (rather than scatters) infrared (IR) light back to its source. An active marker includes an infrared receiver that emits infrared light when the receiver detects an infrared pulse. It is understood that the marker can be affixed to different locations in different application scenarios. For example, the marker can be affixed to an application tool, such as a surgical needle; and also, for example, the marker can be affixed to the surface of the object being implemented, such as... Figure 1As shown, the surface of the embodiment 104 is covered with markers 103, which, exemplarily, include 15 markers.

[0058] In this embodiment, the optical measuring device 101 can be understood as a device for measuring the spatial position information of a marker. Optionally, the optical measuring device 101 may include a position sensor, which may include an infrared light-emitting diode and an IR sensor, etc. For ease of understanding, ... Figure 1 Taking marker 1031 as an example, for instance, marker 1031 is a passive marker. Figure 1 As shown, the optical measuring device 101 generates infrared light through an infrared light-emitting diode and illuminates the marker 1031. The reflective coating on the surface of the marker 1031 reflects the infrared light back to the IR sensor of the optical measuring device 101. The sensor utilizes binocular vision to... Figure 1 The intersection of the light rays shown (i.e., the arrows shown in dashed lines) enables the triangulation of the three-dimensional spatial coordinates of marker 1031.

[0059] It is understood that a communication connection 105 exists between the electronic device 102 and the optical measuring device 101. After the optical measuring device 101 acquires the three-dimensional spatial coordinates of the marker, the electronic device 102 can acquire the coordinate data of the marker through the aforementioned communication connection 105. Optionally, the electronic device 102 includes a display. Therefore, after the electronic device 102 acquires the three-dimensional spatial coordinates of the marker, the marker can be displayed on the display with the coordinate system of the optical measuring device 101 as a reference.

[0060] Understandably, during surgery, the surgical needle is positioned using markers attached to the surface and optical measuring devices. After acquiring the needle's position information through the optical measuring devices, the electronic equipment displays the needle using the coordinate system corresponding to the optical measuring devices. However, on the other hand, after acquiring CT images through a CT scanner, the electronic equipment displays the reconstructed CT images using the coordinate system corresponding to the CT scanner. Because the coordinate systems of the optical measuring devices and the CT scanners are different, misalignment will occur when displayed on the same interface. For example, the surgical needle may actually be 5cm from the patient's body surface, but the monitor may display the needle as if it has already penetrated the patient's body or as if it is displayed far from the patient's body surface. Therefore, the registration (or unification) between the coordinate systems of the optical measuring devices and the CT scanners is a crucial step.

[0061] For example, point cloud registration can be used to register coordinates between different coordinate systems. In this embodiment, a point cloud can be understood as a set of multiple points, and point cloud registration can be understood as solving the rotation and translation matrix between two point clouds to transform the source point cloud to the same coordinate system as the target point cloud. The rotation and translation matrix can be a rigid transform or an Euclidean transform. For example, iterative closest point (ICP) is an important algorithm in point cloud registration, and the ICP algorithm can be expressed by formula (1):

[0062] P t =R·P s +T (1)

[0063] Among them, P t This can be understood as the target point cloud, P s This can be understood as the source point cloud. The ICP algorithm process involves solving for the rotation matrix R and the translation matrix T, such that the point cloud P... s After the above transformations (rotation and translation) and the point cloud P t The results are obtained by overlapping and iterating continuously to minimize the mean square error.

[0064] To better understand this scheme, the ICP algorithm described above will be briefly introduced below. For example, assuming a transformation point set X and a reference point set Y, registration can be performed through the following steps:

[0065] Step 1: Select three points from the transformed point set X, denoted as point X1, point X2, and point X3;

[0066] Step 2: Find three points corresponding to points X1, X2, and X3 from the reference point set Y through exhaustive search, and denote them as points Y1, Y2, and Y3. Translate the center points of points X1, X2, and X3 to the center points of points Y1, Y2, and Y3 to obtain the translation matrix T1 between points X1, X2, and X3 and points Y1, Y2, and Y3. Then, use the singular value decomposition (SVD) method to calculate the rotation matrix R1 between points X1, X2, and X3 and points Y1, Y2, and Y3. Superimpose T1 and R1 to obtain the transformation matrix M1.

[0067] Step 3: Multiply the transformation matrix M1 by the point set X to obtain the transformed point set X' and calculate the registration error, such as the root mean square (RMS) error. If the RMS error is greater than a certain threshold, it is considered that points X1, X2, and X3 are not corresponding point sets with points Y1, Y2, and Y3. Return to step 2 to continue exhaustively searching for corresponding points.

[0068] Step 4: After finding the corresponding point set, sort the points in the transformed point set X' according to their distance from the points in the reference point set Y. Also, the points in the transformed point set X' that are closest to the points in the reference point set Y have the same index. Then record the order of change of the points in the transformed point set X'.

[0069] Step 5: Calculate the rotation matrix R2 based on the transformed point set X' and the reference point set Y, and multiply the rotation matrix R2 by the transformed point set X' to obtain the transformed point set X”.

[0070] Step 6: If the average distance between the transformed point set X” and the reference point set Y is less than the reference threshold, stop the iterative calculation. If the average distance between the transformed point set X” and the reference point set Y is greater than the reference threshold, repeat steps 4 and 5 until the number of iterations exceeds the reference number. If the number of iterations exceeds the reference number, repeat steps 1-5. During the re-execution of steps 1-5, the order of the transformed point set X is adjusted using the change order, so that the order of the corresponding points in the transformed point set X” is the same as the order of the points in the transformed point set X”. The transformation matrix is ​​then calculated using the transformed point set X” with the adjusted order.

[0071] In some embodiments, the registration between the coordinate system corresponding to the optical measurement device and the coordinate system corresponding to the CT device can be achieved through... Figure 2 This can be achieved in the manner shown. Figure 2 This is a schematic diagram of a scenario provided by an embodiment of this application, in which the coordinate system corresponding to the optical measurement device is registered with the coordinate system corresponding to the CT device.

[0072] For example, such as Figure 2 As shown in section 201, five markers were affixed to the surface of the subject's body. Then, the subject was inserted into a CT scanner for a CT scan (e.g., ...). Figure 2 (The 2011 part); then, the above 5 markers are segmented from the scanned CT images, and the coordinates of the above 5 markers are used as the source point cloud (e.g., Figure 2 (The 2012 part). On the other hand, the coordinates of the above 5 markers were collected using optical measurement equipment (such as...). Figure 2 (The 2013 part); then, the coordinates of the above 5 markers collected are used as the target point cloud (e.g., Figure 2 (The 2014 section).

[0073] It should be understood that Figure 2 The source point cloud in the image is obtained based on the coordinate system of the CT equipment. Figure 2 The target point cloud is obtained based on the coordinate system of the optical measurement equipment. For point clouds from different coordinate systems, the ICP algorithm can be used to register the source point cloud to the target point cloud, thereby achieving registration between the coordinate system corresponding to the optical measurement equipment and the coordinate system corresponding to the CT equipment. This is understandable. Figure 2 For example, the coordinates determined by the CT device are used as the source point cloud and the coordinates determined by the optical measurement device are used as the target point cloud for registration. Alternatively, the coordinates determined by the CT device can be used as the target point cloud and the coordinates determined by the optical measurement device can be used as the source point cloud for registration.

[0074] However, in practice, optical measurement equipment may encounter deviations, i.e., artifacts, during the process of acquiring the coordinates of markers. In some scenarios, the surgical environment may contain reflective objects, causing artifacts (also understood as flashes) to be detected by the optical measurement equipment. In other scenarios, artifacts may also occur when the optical measurement equipment calculates the coordinates of the marker using infrared light. For an example, please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram of a scenario where an optical measuring device generates artifact points during the process of acquiring the coordinates of a marker, as provided in an embodiment of this application.

[0075] like Figure 3 As shown, Figure 3 Markers 302 and 303 can be understood as real-world markers, representing the three-dimensional spatial coordinates of markers 302 and 303 that the optical measuring device 301 needs to acquire. It can be understood that when the optical measuring device 301 acquires the coordinates of the markers, it calculates the line between the marker and each IR sensor. For example... Figure 3 As shown, the optical measurement device 301 calculates the lines between marker 302 and IR sensor 3011, the lines between marker 302 and IR sensor 3012, the lines between marker 303 and IR sensor 3011, and the lines between marker 303 and IR sensor 3012, ultimately as follows: Figure 3 As shown by the dashed line in the image.

[0076] When the lines between the aforementioned markers and the IR sensor intersect or nearly intersect, the optical measuring device 301 will calculate the distance between the intersecting or nearly intersecting lines. If the distance is less than a certain threshold (e.g., 2 mm), the optical measuring device 301 will consider the intersection of the two lines as a marker as well. Figure 3 The white spherical marker in the middle. However, it can be understood that, as Figure 3 The white spherical markers in the image are not real markers, but rather artifacts.

[0077] In the above-mentioned situations, the markers collected by the optical measurement equipment may not be actual markers pasted on the surface of the object, resulting in a time-consuming and inefficient registration process between the source and target point clouds. Based on these problems, embodiments of this application provide a method and related apparatus for determining coordinates. This application can save time spent in the registration process and improve registration efficiency. It is understood that the coordinate determination method provided in this application can be executed by a coordinate determination device, which can be any electronic device capable of executing the technical solutions disclosed in the method embodiments of this application. For example, the coordinate determination device can be a computer, tablet computer, desktop computer, etc., and this application does not limit it to this. It should also be understood that the method embodiments in this application can also be implemented by a processor executing computer program code.

[0078] The embodiments of this application are described below with reference to the accompanying drawings. For example, please refer to the accompanying drawings. Figure 4 , Figure 4 This is a flowchart illustrating a method for determining coordinates provided in an embodiment of this application, as shown below. Figure 4 As shown, the above method includes:

[0079] 401: Determine a reference plane, which is obtained from the first coordinates of every three markers out of N markers at a reference time. These first coordinates are acquired by an optical measuring device, and N is an integer greater than or equal to 3.

[0080] In this application embodiment, the aforementioned marker can also be referred to as an optical marker ball, a reflective marker, a reflective marker point (mark point), etc., and this application does not impose any limitation on this. Furthermore, the aforementioned marker can be an active marker or a passive marker; for an explanation of active and passive markers, please refer to the preceding text. Figure 1 The relevant descriptions will not be repeated here. In this embodiment, the above-mentioned N markers can be understood as real markers existing in the actual scene. Depending on the different scenes, the above-mentioned N markers can be pasted in different locations. For example, the above-mentioned N markers can be pasted on the body surface of the subject, such as the chest, abdomen, etc.

[0081] In this embodiment, the optical measuring device can use infrared light to acquire the coordinates of the marker; the specific acquisition method can be found in the preceding text. Figure 1 The relevant descriptions will not be repeated here. It should be understood that the coordinates of the markers collected by the optical measuring equipment are three-dimensional spatial coordinates. That is to say, the first coordinate mentioned above, as well as the second coordinate in step 402 below, are all three-dimensional spatial coordinates.

[0082] In this embodiment, the aforementioned reference time can be understood as the time when the optical measuring device accurately acquires the coordinates of each of the N markers. That is, when the optical measuring device acquires the coordinates of the N markers at the aforementioned reference time, it can accurately acquire the N coordinates, and each of the N coordinates corresponds one-to-one with each of the N markers. For example, when the object is in a stable state, the optical measuring device can be adjusted to accurately acquire the coordinates of each marker.

[0083] It is understandable that, since the surface of the object being measured is not a smooth plane, and in real-world scenarios, the markers on the object's surface are generally evenly and staggered, the coordinates of every three markers among the aforementioned N markers can form a plane. Therefore, after the optical measuring device acquires the N coordinates of the aforementioned N markers, it can obtain the aforementioned reference plane. It is understandable that, since N is greater than or equal to 3, when N equals 3, the aforementioned reference plane is a single plane; when N is greater than 3, the aforementioned reference plane should be understood as a set of multiple planes. For example, when N equals 5, the optical measuring device can acquire 5 first coordinates using infrared light. Based on these 5 first coordinates, 10 (i.e., ...) can be obtained. The above 10 planes can be understood as the above reference planes, that is, the above reference planes can be understood as a set of multiple planes.

[0084] In one implementation of determining a reference plane, the coordinate determining device can establish a communication connection with the optical measuring device to obtain the first coordinates of each of the N markers collected by the optical measuring device at a reference time, and then derive the reference plane based on the first coordinates.

[0085] In another implementation of determining the reference plane, the reference plane can be determined by other electronic devices (in some embodiments, it can be an optical measuring device), and then the coordinate determining device can establish a communication connection with other electronic devices to determine the reference plane.

[0086] 402: Obtain the M second coordinates of the N markers at the first moment. The second coordinates are acquired by the optical measurement device. M is an integer greater than or equal to 3. The first moment is later than the reference moment.

[0087] In this step, the second coordinates of the aforementioned N markers at the first moment are also acquired by the optical measurement device described in step 401. It is understood that in practical applications (e.g., during surgery), after the markers are attached to the patient's body surface, the optical measurement device will continuously acquire the coordinates of the markers for a period of time. In the above scenario, the aforementioned first moment can be understood as any moment during the surgical procedure. As mentioned above... Figure 3 As can be understood from the relevant description, deviations may occur during the process of the optical measuring device acquiring the coordinates of the markers, resulting in the optical measuring device obtaining M second coordinates when acquiring the coordinates of the N markers at the first moment. It should be understood that since two points form a line segment, and a line segment can rotate, when using a registration algorithm to register different coordinates (or can be understood as a set of points), at least 3 coordinates are required for registration respectively. Therefore, in the embodiments of this application, the above M is greater than or equal to 3.

[0088] For example, in one possible implementation, the coordinate determining device can establish a communication connection with an optical measuring device to obtain the M second coordinates of the N markers at a first moment.

[0089] In some embodiments, Figure 4 In the method shown, step 401 includes: in response to a first triggering operation, determining the reference plane, wherein the first triggering operation is used to indicate that the implementation object is in a first state, and the surface of the implementation object is covered with the N markers;

[0090] Step 402 includes: in response to a second triggering operation, obtaining the M second coordinates of the N markers at the first time point, wherein the second triggering operation is used to indicate that the implementation object is in a second state, and the stability level corresponding to the first state is greater than the stability level corresponding to the second state.

[0091] In this embodiment, the first triggering operation can be understood as a user operation (e.g., a doctor's or other operator's operation). For example, the coordinate determining device may include an input component, such as a keyboard, mouse, touchscreen, or touchpad, which can be used to implement the triggering operation. For example, the triggering operation can be implemented by clicking a software button with a mouse or by clicking a software button on a touchscreen.

[0092] In this embodiment, after the first triggering operation is triggered, the coordinate determining device determines the reference plane. This can be understood as follows: in response to the first triggering operation, the coordinate determining device determines the reference plane based on the acquired coordinates; or, in response to the first triggering operation, the coordinate determining device uses the acquired plane as the reference plane. For example, in response to a click operation on a first button (e.g., a "reference data acquisition" button), the coordinate determining device acquires N coordinates of the N markers at a reference time from the optical measuring device and obtains the reference plane based on the N coordinates. In the above case, the click operation on the first button can be understood as the first triggering operation.

[0093] Similarly, the second triggering operation described above can be understood as a user operation (e.g., a user operation by a doctor or other operator). For example, the coordinate determining device may include an input component, such as a keyboard, mouse, touchscreen, or touchpad, which can be used to implement the triggering operation. For example, the triggering operation can be implemented by clicking a software button with a mouse, or by clicking a software button on a touchscreen.

[0094] In this embodiment, after the second triggering operation is triggered, the coordinate determining device acquires M second coordinates of the N markers at the first moment. For example, in response to a click on a second button (e.g., a "Reference Data Acquisition Complete" button, a "Surgery Start" button, etc.), the coordinate determining device acquires coordinate data from the optical measuring device and uses this coordinate data as the M second coordinates of the N markers at the first moment. In this case, the click on the second button can be understood as the second triggering operation.

[0095] In this embodiment, the first triggering operation is used to indicate that the implementation object is in a first state, and the second triggering operation is used to indicate that the implementation object is in a second state. It can be understood that the reference plane is obtained based on the coordinate data of the N markers at the reference time, and the M second coordinates are the coordinate data of the N markers at the first time. That is, the implementation object is in the first state at the reference time, and the implementation object is in the second state at the first time. The stability level corresponding to the first state is greater than the stability level corresponding to the second state. In this embodiment, using the data collected by the implementation object in the more stable first state as reference data allows for more accurate subsequent registration.

[0096] It is understandable that the aforementioned N markers are affixed to the subject's body surface. The subject's breathing and other limb movements will affect the optical measurement equipment's coordinate measurement of the markers, causing a significant difference between the real-time distribution of the markers and the distribution at the aforementioned reference time (e.g., the time when the subject is scanned using a CT scanner). Regarding the stability corresponding to the two states, it can be understood that the subject's body activity amplitude in the first state is less than that in the second state. For example, the subject's breath-holding state can be considered the first state, and the subsequent free breathing state can be considered the second state.

[0097] 403: When M is greater than N, at least three coordinates for registration are determined from the M second coordinates, the angle between the first plane and the second plane is less than or equal to a first threshold, the first plane is a plane obtained based on three of the at least three coordinates for registration, and the second plane is one of the reference planes.

[0098] It should be understood that, in the event of artifacts appearing during the acquisition of the coordinates of the markers by the optical measuring device at the aforementioned first moment, the number M of the second coordinates acquired at the aforementioned first moment is greater than the number N of the markers actually pasted on the user's body surface. In this embodiment, registration can be understood as spatial registration, that is, integrating and displaying the surgical instruments (e.g., surgical needles) and the real-time markers on the patient's body surface during the surgery with the preoperative CT model of the patient, thereby effectively guiding the surgical plan and directing the surgery in real time. It should be understood that the aforementioned preoperative CT model may include a CT-reconstructed surgical area, such as a three-dimensional model of the skin, bones, or lungs.

[0099] In this embodiment, the coordinates used for registration can be understood as coordinates used for registration with a third coordinate, which is obtained by segmenting the N markers in the CT image. The CT image is acquired by the CT device at the reference time. The registration process and other descriptions of the coordinates used for registration in this embodiment can be found in the following text. Figure 7 and Figure 8 Corresponding implementation examples.

[0100] It is understood that in this embodiment, the angle between the two planes ranges from [0, π / 2], and the angle between the planes can be determined by the normal vector of the plane. For example, after obtaining the M second coordinates, the coordinate determining device can form a candidate plane from any three of the M second coordinates, and then calculate the angle A between the two planes by calculating the normal vector A of the candidate plane and the normal vector B of the second plane. If the angle A is less than or equal to the first threshold, the three coordinates corresponding to the candidate plane are used as coordinates for registration. In this case, the candidate plane can be understood as the first plane. It should be understood that the first threshold can be adjusted according to actual conditions. For example, the first threshold can be 3°, 5°, 7°, etc., and this application does not limit it.

[0101] It should be understood that since the aforementioned second plane can be interpreted as one of the aforementioned reference planes, that is, the candidate plane formed by three of the aforementioned M second coordinates only needs to have an angle between it and one of the aforementioned reference planes that is less than or equal to the aforementioned second threshold. If the angle between the plane formed by three of the M second coordinates actually acquired at the first moment and the plane formed by the N reference coordinates acquired at the reference moment is less than or equal to the aforementioned first threshold, it can be considered that the structural features corresponding to the three coordinates acquired at the first moment are very similar to the structural features corresponding to the three coordinates at the reference moment. Therefore, it can be considered that the three coordinates acquired at the first moment are the coordinates of the actual marker.

[0102] In this embodiment, a reference plane is first determined. This reference plane is obtained based on the first coordinates of every three markers out of N markers collected by the optical measuring device at a reference time. Since the user is in a stable state at the reference time, the coordinates of the N markers collected by the optical measuring device at the reference time can be considered accurate, i.e., the coordinates collected by the optical measuring device are the true coordinates. After the reference time, the coordinate determining device acquires M second coordinates of the N markers collected by the optical measuring device at the first time. At this time, artifacts in the actual scene will cause M to be greater than N. The coordinate determining device determines at least three coordinates for registration from the M second coordinates. The angle between the first plane and the second plane is less than or equal to a first threshold. The first plane is a plane obtained based on three of the at least three coordinates for registration, and the second plane is one of the reference planes. That is, the plane formed by the coordinates used for registration almost coincides with the reference plane. When the two planes almost overlap, the structural features of the three coordinates corresponding to the two planes are almost the same. Since the coordinates collected at the reference time are the real coordinates, the above method can effectively eliminate artifact points, determine the real coordinates for registration, thereby saving registration time and improving the efficiency and accuracy of registration.

[0103] It is understandable that when artifacts appear in the coordinates of the marker acquired by the optical measurement equipment, M is greater than N. It should be understood that in some embodiments, when M is equal to N, or when M is less than N, the coordinate determining device can also execute the coordinate determining method to determine the truly existing coordinates among the M second coordinates, thereby improving the efficiency and accuracy of registration.

[0104] In some embodiments, Figure 4 In the method shown, the absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to the second threshold. The first figure is a triangle formed by the three coordinates corresponding to the first plane, and the second figure is a triangle formed by the three coordinates corresponding to the second plane.

[0105] Alternatively, the absolute value of the difference between the area of ​​the first graphic and the area of ​​the second graphic is less than or equal to the third threshold.

[0106] In this step, the second plane should be understood as a plane whose angle with the first plane is less than or equal to the first threshold. In this embodiment, the coordinates corresponding to the plane (including the first plane and the second plane) can be understood as the three coordinates that make up the plane. For example, coordinates A, B, and C make up plane A, and the coordinates corresponding to plane A can be understood as coordinates A, B, and C. Exemplarily, the three coordinates used for registration are coordinates A, B, and C. In this embodiment, the angle between plane A calculated by coordinates A, B, and C and plane B in the reference plane is less than the first threshold; at the same time, the absolute value of the difference between the perimeter A of the triangle calculated by coordinates A, B, and C and the perimeter B of the triangle calculated by the three coordinates corresponding to plane B is less than the second threshold, or the absolute value of the difference between the area A of the triangle calculated by coordinates A, B, and C and the area B of the triangle calculated by the three coordinates corresponding to plane B is less than the third threshold.

[0107] In this embodiment, the second threshold and the third threshold can be set according to the actual situation. For example, the second threshold can be 3 mm, 4 mm, etc., and the third threshold can be 3 square millimeters, 4 square millimeters, etc. This application does not limit them.

[0108] In this embodiment, a normal vector can be calculated based on every three coordinates, and the perimeter of a triangle can also be calculated. Therefore, if the angle between the plane obtained from the three coordinates collected in real time and the second plane is less than or equal to the first threshold, and the absolute value of the difference between the perimeter of the triangle obtained from the three coordinates and the perimeter of the triangle formed by the three coordinates corresponding to the second plane is less than the second threshold, using these three coordinates as coordinates for registration can further effectively eliminate artifact points, and then use the coordinates corresponding to the real markers for registration, saving registration time and improving the efficiency and accuracy of registration.

[0109] In some embodiments, the coordinate determining device can also be... Figure 5 The method shown determines the coordinates used for registration, where, Figure 5 This is a flowchart illustrating another method for determining coordinates provided in an embodiment of this application, as shown below. Figure 5 As shown, the above method includes:

[0110] 501: Determine a reference plane, which is obtained from the first coordinates of every three markers out of N markers at a reference time. These first coordinates are acquired by an optical measurement device, and N is an integer greater than or equal to 3.

[0111] 502: Obtain the M second coordinates of the N markers at the first moment. The second coordinates are acquired by the optical measurement device. M is an integer greater than or equal to 3. The first moment is later than the reference moment.

[0112] For steps 501 and 502 above, please refer to the relevant descriptions of steps 401 and 402 above, which will not be repeated here.

[0113] 503: From the M second coordinates, three coordinates for registration are determined. The angle between the plane corresponding to the three coordinates for registration and the second plane is less than or equal to the first threshold. The second plane is one of the planes in the reference plane.

[0114] 504: Combine the remaining coordinates with two of the three coordinates used for registration to form a third plane. The remaining coordinates are any coordinates among the M second coordinates other than the three coordinates used for registration.

[0115] 505: If the angle between the third plane and the second plane is less than or equal to the first threshold, the remaining coordinates shall be used as the coordinates for registration.

[0116] To facilitate understanding of steps 503-505 above, please refer to the example provided. Figure 6 , Figure 6This is a flowchart illustrating yet another method for determining coordinates provided in this application. For example... Figure 6 As shown, the method includes:

[0117] 601: Select coordinates A1, A2, and A3 from M second coordinates.

[0118] In this step, the coordinate determining device can randomly select three coordinates from M second coordinates as coordinates A1, coordinate A2, and coordinate A3.

[0119] 602: Form plane A by aligning coordinates A1, A2, and A3.

[0120] 603: Determine whether the angle between plane A and the second plane is less than the first threshold.

[0121] The second plane in this step can be understood as the second plane in step 403 above, and the first threshold in this step can be understood as the first threshold in step 403 above.

[0122] If the judgment result in step 603 is "yes", the coordinate determination device executes step 604: using coordinates A1, coordinates A2 and coordinates A3 as coordinates for registration.

[0123] In this step, after using coordinates A1, A2, and A3 as coordinates for registration, coordinates A1, A2, and A3 can be understood as the three coordinates used for registration in step 503 above.

[0124] If the judgment result of step 603 is "no", the coordinate determining device will re-execute step 601. It should be understood that when the coordinate determining device re-executes step 601, it will re-select three coordinates from the M second coordinates.

[0125] 605: Place coordinates A1, A2, and A... i Forming plane B, the coordinates A i It is any coordinate other than coordinates A1, A2, and A3 among the M second coordinates.

[0126] Coordinate A in this step i This can be understood as the remaining coordinates in step 504 above; coordinates A1 and A2 can be understood as two of the three coordinates used for registration in step 505 above; plane B can be understood as the third plane in step 504 above. It should be understood that... Figure 6 The method shown connects coordinates A1, coordinates A2, and A... i By combining these coordinates, plane B can be obtained. In some embodiments, coordinates A1 and A3, or coordinates A2 and A3, can also be selected. iCombining these elements yields plane B.

[0127] 606: Determine whether the angle between plane B and the second plane is less than the first threshold.

[0128] The second plane in this step can be understood as the second plane in step 403 above, and the first threshold in this step can be understood as the first threshold in step 403 and step 603 above.

[0129] If the judgment result in step 606 is "yes", the coordinate determining device executes step 607: set coordinate A i As coordinates used for registration.

[0130] If the judgment result in step 606 is "no", the coordinate determination device executes step 608: determine whether all coordinates except coordinates A1, coordinates A2 and coordinates A3 in the M second coordinates have been traversed.

[0131] If the result of step 608 is "yes", the method ends. If the result of step 608 is "no", the coordinate determining device re-executes step 605. It should be understood that when the coordinate determining device re-executes step 605, it will reselect coordinate A. i .

[0132] In this embodiment, after determining three coordinates for registration from the above M second coordinates, other coordinates for registration are determined by combining the three coordinates for registration with each of the remaining coordinates to form a plane. This can improve the efficiency of the coordinate determining device in determining the coordinates for registration, thereby improving the efficiency of registration.

[0133] In some embodiments, Figure 5 In the method shown, step 505 includes:

[0134] If the absolute value of the difference between the perimeter of the third figure and the perimeter of the second figure is less than or equal to the second threshold, the coordinates corresponding to the third plane are used as coordinates for registration, and the third figure is a triangle formed by the three coordinates corresponding to the third plane.

[0135] Alternatively, if the absolute value of the difference between the area of ​​the third graphic and the area of ​​the second graphic is less than or equal to the third threshold, the coordinates corresponding to the third plane are used as the coordinates for registration.

[0136] This embodiment can be understood as, in Figure 6 In step 606 of the method shown, the coordinates corresponding to plane B (i.e., coordinates A1, coordinate A2, and coordinate A) are further considered. iThe difference in perimeter between the triangle formed by the first triangle and the triangle corresponding to the second plane. In this embodiment, the coordinate determining device is in... Figure 6 Based on the method shown, the coordinates used for registration can be further determined by the perimeter or area of ​​the triangle formed by the corresponding coordinates of the plane. This not only effectively eliminates artifact points, but also further improves the efficiency of determining the coordinates used for registration.

[0137] The coordinate determination method provided in the embodiments of this application has been described above. Next, the registration process in the embodiments of this application will be described. The registration in the embodiments of this application is the registration between the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device. To better understand the registration process of this solution, the specific registration process will be explained below.

[0138] For example, please refer to Figure 7 , Figure 7 This is a schematic flowchart illustrating a registration method provided in an embodiment of this application. It should be understood that... Figure 7 The registration method shown can be performed by the coordinate determining device described above. It should also be understood that, as... Figure 7 In the registration method shown, multiple markers are pasted on the surface of the object. In this embodiment, the registration between the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device is achieved based on the coordinates of the multiple markers under the CT device and the coordinates of the multiple markers under the optical measurement device. Therefore, the coordinate determination device will process the data collected by the CT device and the optical measurement device respectively. The processing of the data collected by the CT device by the coordinate determination device includes steps 701-704, and the processing of the data collected by the optical measurement device includes steps 705-706.

[0139] It should be noted that the numbering before each step in this embodiment is for ease of understanding of the scheme and does not limit the execution order of each step.

[0140] Furthermore, it should be understood that in real-world scenarios, coordinate system registration is a real-time process. Taking a surgical procedure as an example, the two coordinate systems mentioned above need to be registered throughout the entire surgical process to effectively guide the surgical operation. Therefore, for ease of understanding, as... Figure 7 As shown, the above registration method is divided into a first stage and a second stage. The first stage can be understood as registration before surgery, and the second stage can be understood as registration during surgery.

[0141] Next, combine Figure 7 The first stage described above describes the processing of data acquired by the CT equipment by the coordinate determination device. This processing includes:

[0142] 701: Acquire multiple CT images captured by the CT equipment at time A.

[0143] In this step, the aforementioned time A can be understood as the reference time in step 401 above. For example, the coordinate determining device can establish a communication connection with the CT equipment and obtain multiple CT images of the implementation object at time A through the aforementioned communication connection.

[0144] 702: Three-dimensional reconstruction based on these multiple CT images.

[0145] In this step, the coordinate determining device segments skin tissue and bone contours from the multiple CT images to achieve three-dimensional reconstruction of the internal structure of the surgical area of ​​the subject. Optionally, the coordinate determining device includes a display, so the coordinate determining device can display the internal structure of the subject in the form of a three-dimensional view on the display.

[0146] 703: Determine the coordinates of multiple markers from the multiple CT images to obtain coordinate A.

[0147] It should be understood that multiple markers are affixed to the surface of the subject, and the multiple CT images obtained by CT scanning the subject using a CT scanner include these multiple markers. In this step, the coordinate determining device segments the multiple markers from the multiple CT images to obtain the coordinates A of each marker in the coordinate system corresponding to the CT scanner. In this step, any coordinate A can be understood as the aforementioned third coordinate.

[0148] It is understandable that when performing three-dimensional reconstruction based on the above-mentioned multiple CT images in step 702, multiple markers in the above-mentioned multiple CT images can be displayed in the reconstructed three-dimensional view.

[0149] 704: Take coordinate A as point cloud A.

[0150] In this step, it should be understood that the coordinate A mentioned above refers to the coordinates of each of the multiple markers in the coordinate system corresponding to the CT equipment. That is, the coordinate A mentioned above can be understood as a collective term for multiple coordinates.

[0151] The following describes the processing of data acquired by the CT equipment by the coordinate determination device in the first stage described above. This processing includes:

[0152] 705: Obtain the coordinates of the multiple markers collected by the optical measuring device at time A, and obtain coordinates B.

[0153] In this step, the optical measuring equipment acquires the coordinates of multiple markers attached to the surface of the object. For specific acquisition methods, please refer to the previous text. Figure 1The relevant description of step 401 above will not be repeated here. The coordinate B mentioned above can be understood as the first coordinate in step 401 above.

[0154] 706: Use coordinate B as point cloud B.

[0155] In this step, it should be understood that the coordinate B mentioned above refers to the coordinates of each of the multiple markers in the coordinate system corresponding to the optical measuring device. That is, the coordinate B mentioned above can be understood as a collective term for multiple coordinates.

[0156] After determining the point cloud A and the point cloud B, the coordinate determination device performs step 707: registering the point cloud A and the point cloud B based on the registration algorithm to obtain the registration matrix M0.

[0157] It should be understood that the coordinates in point cloud A are obtained based on the coordinate system corresponding to the CT device, and the coordinates in point cloud B are obtained based on the coordinate system corresponding to the optical measurement device. The registration between the two different coordinate systems can be performed using the ICP algorithm to obtain the registration matrix M0.

[0158] It should be understood that a point cloud is a collection of multiple points. In the embodiments of this application, each marker can be understood as a point, and the multiple markers can be combined to form a point cloud. However, in the process of processing each point (that is, each marker), the coordinate determination device converts the coordinates of each point into a matrix for processing. In other words, registration can be understood as the registration between two point clouds. Registration is achieved by converting the coordinate set of the point cloud into matrix form and performing operations and transformations. A point is an entity, and coordinates are the spatial position expression of a point. Therefore, it can also be understood as the registration between multiple coordinates, such as the registration between coordinate A and coordinate B, and the registration between coordinate A and coordinate C.

[0159] After obtaining the above registration matrix M0, it can also be understood that the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device have been registered at time A. That is, in addition to the above-mentioned multiple markers, other objects in the coordinate system corresponding to the CT device, such as skin, bones and lung organs obtained from the above-mentioned multiple CT images, can be transformed into the coordinate system corresponding to the optical measurement device through the above-mentioned registration matrix M0.

[0160] Steps 701-707 in the first stage described above can be understood as the registration between coordinate systems at time A. In subsequent processes, the breathing and other body movements of the subject will cause changes in the coordinates of the multiple markers acquired by the optical measurement equipment. Therefore, after time A, the coordinate determination device needs to reacquire the coordinates of the multiple markers to achieve real-time registration between coordinate systems. For ease of understanding, the registration after time A is referred to as the registration in the second stage. Figure 7 As shown, the second-stage registration process includes:

[0161] 708: Obtain the coordinates of the multiple markers collected by the optical measuring device at time B, and obtain coordinate C.

[0162] In this step, time B is later than time A, and time B can be understood as the first time in step 402. Coordinate C can be understood as the M second coordinates in step 402.

[0163] 709: Use the coordinates C as the point cloud C.

[0164] In this step, it should be understood that the coordinate C mentioned above refers to the coordinates of each of the multiple markers in the coordinate system corresponding to the optical measuring device. That is, the coordinate C mentioned above can be understood as a collective term for multiple coordinates.

[0165] 710: Based on the registration algorithm, point cloud B and point cloud C are registered to obtain the registration matrix M. h .

[0166] In this step, the coordinates B in point cloud B and C in point cloud C are coordinates acquired by the optical measuring device at different times. That is, coordinate B is the coordinate acquired at time A, and coordinate C is the coordinate acquired at time B. Therefore, the registration matrix M... h This can be understood as registering the positions and attitudes of the aforementioned markers at time B to the positions and attitudes at time A using a registration matrix. The registration algorithm in this step can also be the ICP algorithm described above.

[0167] 711: Based on M0 and M h The registration matrix M between coordinate C and coordinate A at time A is obtained.

[0168] 712: Based on the registration matrix M, register the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device.

[0169] To facilitate understanding of the above steps, please refer to the example provided. Figure 8 , Figure 8This is a schematic diagram illustrating the registration of the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device at time B, as provided in an embodiment of this application.

[0170] like Figure 8 As shown, the registration matrix M0 is the registration matrix of the two coordinate systems obtained through coordinates A and B at time A. h The registration matrix for the position and orientation of the marker at times A and B, obtained using coordinates B and C, is thus calculated by performing the operation M0×M. h The obtained matrix M can be used as the registration matrix of the two coordinate systems at time B. It should be understood that the two coordinate systems mentioned above refer to the coordinate system corresponding to the CT equipment and the coordinate system corresponding to the optical measurement equipment.

[0171] It should be understood that, through the above registration matrix M, an object (such as a surgical needle) in the coordinate system corresponding to the optical measurement device can be registered to the coordinate system corresponding to the CT device. In some embodiments, by performing an inverse operation on the above registration matrix M, an object in the coordinate system corresponding to the CT device can be registered to the coordinate system corresponding to the optical measurement device.

[0172] Based on the above Figure 7 and Figure 8 In the relevant descriptions and embodiments of this application, the coordinates used for registration can be understood as coordinates used for registration with the aforementioned coordinates A. For example, Figure 7 In the method shown, coordinates B and C can both be understood as coordinates used for registration. Based on the relevant descriptions of the above embodiments, it can be understood that artifacts may appear in coordinates C acquired by the optical measuring device at time B. That is, the coordinate determining device needs to determine the coordinates used for registration from coordinates C. Therefore, the coordinates used for registration determined from the M second coordinates should be understood as the coordinates determined from the above coordinates C for registration with coordinate A.

[0173] The methods provided in the embodiments of this application have been described in detail above. The apparatus provided in the embodiments of this application is described below.

[0174] Please see Figure 9 , Figure 9 This is a schematic diagram of a coordinate determination device provided in an embodiment of this application. The coordinate determination device 90 is used to execute the aforementioned coordinate determination method. It should be understood that any device capable of implementing the coordinate determination method provided in this application falls within the protection scope of this application. Exemplarily, the coordinate determination device 90 can be a mobile phone, desktop computer, or portable laptop, etc., and this embodiment of the application does not limit it. Figure 9As shown, the coordinate determining device 90 includes a determining unit 901 and an acquiring unit 902. Optionally, the coordinate determining device 90 may further include a responding unit 903 and a constituting unit 904. The descriptions of each unit are as follows:

[0175] The determining unit 901 is used to determine a reference plane, which is obtained based on the first coordinates of every three markers out of N markers at a reference time. The first coordinates are acquired by an optical measuring device, and N is an integer greater than or equal to 3.

[0176] The acquisition unit 902 is used to acquire M second coordinates of the N markers at a first moment, wherein the second coordinates are acquired by the optical measurement device, M is an integer greater than or equal to 3, and the first moment is later than the reference moment.

[0177] The determining unit 901 is further configured to determine at least three coordinates for registration from the M second coordinates, wherein the angle between the first plane and the second plane is less than or equal to a first threshold, the first plane is a plane obtained based on three of the at least three coordinates for registration, and the second plane is one of the reference planes.

[0178] In one possible implementation, the above-mentioned device further includes a response unit 903, which is used to respond to a first trigger operation and control the determination unit 901 to determine the reference plane. The first trigger operation is used to indicate that the implementation object is in a first state, and the surface of the implementation object is covered with the above-mentioned N markers.

[0179] The response unit 903 is also used to respond to the second trigger operation and control the acquisition unit to acquire the M second coordinates of the N markers at the first time. The second trigger operation is used to indicate that the implementation object is in a second state, and the stability of the first state is greater than the stability of the second state.

[0180] In one possible implementation, the absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to a second threshold, wherein the first figure is a triangle formed by three coordinates corresponding to the first plane, and the second figure is a triangle formed by three coordinates corresponding to the second plane.

[0181] Alternatively, the absolute value of the difference between the area of ​​the first graphic and the area of ​​the second graphic is less than or equal to the third threshold.

[0182] In one possible implementation, the determining unit 901 is specifically used to determine three coordinates for registration from the M second coordinates, wherein the angle between the plane corresponding to the three coordinates for registration and the second plane is less than or equal to the first threshold.

[0183] The above-mentioned device also includes a component unit 904, which is used to form a third plane by combining the remaining coordinates with two of the three coordinates used for registration, wherein the remaining coordinates are any coordinates other than the three coordinates used for registration among the M second coordinates.

[0184] The determining unit 901 is specifically used to use the remaining coordinates as coordinates for registration when the angle between the third plane and the second plane is less than or equal to the first threshold.

[0185] In one possible implementation, the determining unit 901 is specifically used to use the coordinates corresponding to the third plane as coordinates for registration when the absolute value of the difference between the perimeter of the third figure and the perimeter of the second figure is less than or equal to the second threshold. The third figure is a triangle formed by the three coordinates corresponding to the third plane.

[0186] Alternatively, the determining unit 901 is specifically used to use the coordinates corresponding to the third plane as coordinates for registration when the absolute value of the difference between the area of ​​the third graphic and the area of ​​the second graphic is less than or equal to the third threshold.

[0187] In one possible implementation, the at least three coordinates used for registration are used to register with a third coordinate, which is obtained by segmenting the N markers in the computed tomography (CT) image, which is acquired by the CT device at the reference time.

[0188] Please see Figure 10 , Figure 10 This is a schematic diagram of another coordinate determination device provided in an embodiment of this application. This coordinate determination device 100 can be used to implement the above-described coordinate determination method. For example, the coordinate determination device 100 can be a mobile phone, desktop computer, laptop, or other similar device.

[0189] like Figure 10 As shown. The coordinate determining device 100 includes at least one processor 1002 and a transceiver 1001 for communicating with other devices / appliances via a transmission medium. The processor 1002 can use the transceiver 1001 to send and receive data and / or signaling.

[0190] Optionally, the coordinate determination device 100 may further include at least one memory 1003 for storing program instructions and / or data. The memory 1003 is coupled to the processor 1002. The coupling in this embodiment is an indirect coupling or communication connection between devices, units, or modules, and can be electrical, mechanical, or other forms, used for information exchange between devices, units, or modules. The processor 1002 may operate in conjunction with the memory 1003. The processor 1002 may execute program instructions stored in the memory 1003. At least one of the at least one memory may be included in the processor.

[0191] This application embodiment does not limit the specific connection medium between the transceiver 1001, processor 1002, and memory 1003. This application embodiment... Figure 10 The memory 1003, processor 1002, and transceiver 1001 are connected via a bus 1004. Figure 10 The connections between other components are shown in thick lines only and are not intended to be limiting. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0192] In the embodiments of this application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0193] It is understood that when the coordinate determining device 100 is the aforementioned coordinate determining device 90, the actions performed by the acquisition unit 902 can be performed by the transceiver 1001, or by the processor 1002; the actions performed by the determining unit 901, the response unit 903, and the component unit 904 can be performed by the processor 1002.

[0194] This application also provides a computer-readable storage medium storing computer code that, when executed on a computer, causes the computer to perform the methods described in the above embodiments.

[0195] This application also provides a computer program product, which includes computer code or a computer program that, when run on a computer, causes the methods described in the above embodiments to be executed.

[0196] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the above claims.

Claims

1. A method for determining coordinates, characterized in that, The method includes: A reference plane is determined, which is obtained based on the first coordinates of every three markers out of N markers at a reference time. The first coordinates are acquired by an optical measurement device, and N is an integer greater than or equal to 3. Obtain M second coordinates of the N markers at a first time point, wherein the second coordinates are acquired by the optical measurement device, and M is an integer greater than or equal to 3, and the first time point is later than the reference time point; When M is greater than N, at least three coordinates for registration are determined from the M second coordinates, the angle between the first plane and the second plane is less than or equal to a first threshold, the first plane is a plane obtained based on three of the at least three coordinates for registration, and the second plane is one of the reference planes; Determining at least three coordinates for registration from the M second coordinates includes: determining three coordinates for registration from the M second coordinates, wherein the angle between the plane corresponding to the three coordinates for registration and the second plane is less than or equal to the first threshold; forming a third plane by combining the remaining coordinates with two of the three coordinates for registration, wherein the remaining coordinates are any coordinates from the M second coordinates other than the three coordinates for registration; and using the remaining coordinates as the coordinates for registration if the angle between the third plane and the second plane is less than or equal to the first threshold.

2. The method according to claim 1, characterized in that, The determination of the reference plane includes: In response to a first triggering operation, the reference plane is determined, the first triggering operation being used to indicate that the implementation object is in a first state, and the N markers are pasted on the body surface of the implementation object; The step of obtaining the M second coordinates of the N markers at the first moment includes: In response to a second triggering operation, the M second coordinates of the N markers at the first time are obtained. The second triggering operation is used to indicate that the implementation object is in a second state, and the stability of the first state is greater than the stability of the second state.

3. The method according to claim 1 or 2, characterized in that, The absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to the second threshold. The first figure is a triangle formed by the three coordinates corresponding to the first plane, and the second figure is a triangle formed by the three coordinates corresponding to the second plane. Alternatively, the absolute value of the difference between the area of ​​the first graphic and the area of ​​the second graphic is less than or equal to the third threshold.

4. The method according to claim 3, characterized in that, The step of using the coordinates corresponding to the third plane as coordinates for registration includes: If the absolute value of the difference between the perimeter of the third figure and the perimeter of the second figure is less than or equal to the second threshold, the coordinates corresponding to the third plane are used as coordinates for registration, and the third figure is a triangle formed by the three coordinates corresponding to the third plane. Alternatively, if the absolute value of the difference between the area of ​​the third graphic and the area of ​​the second graphic is less than or equal to the third threshold, the coordinates corresponding to the third plane are used as the coordinates for registration.

5. The method according to any one of claims 1, 2, and 4, characterized in that, The at least three coordinates used for registration are used to register with a third coordinate, which is obtained by segmenting the N markers in the computed tomography (CT) image, which is acquired by the CT device at the reference time.

6. A coordinate determining device, characterized in that, The device includes: A determining unit is used to determine a reference plane, which is obtained based on the first coordinates of every three markers out of N markers at a reference time. The first coordinates are acquired by an optical measuring device, and N is an integer greater than or equal to 3. The acquisition unit is used to acquire M second coordinates of the N markers at a first time point, wherein the second coordinates are acquired by the optical measurement device, and M is an integer greater than or equal to 3, and the first time point is later than the reference time point; The determining unit is further configured to determine at least three coordinates for registration from the M second coordinates, wherein the angle between the first plane and the second plane is less than or equal to a first threshold, the first plane is a plane obtained based on three coordinates from the at least three coordinates for registration, and the second plane is one of the reference planes; Determining at least three coordinates for registration from the M second coordinates includes: determining three coordinates for registration from the M second coordinates, wherein the angle between the plane corresponding to the three coordinates for registration and the second plane is less than or equal to the first threshold; forming a third plane by combining the remaining coordinates with two of the three coordinates for registration, wherein the remaining coordinates are any coordinates from the M second coordinates other than the three coordinates for registration; and using the remaining coordinates as the coordinates for registration if the angle between the third plane and the second plane is less than or equal to the first threshold.

7. The apparatus according to claim 6, characterized in that, The absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to the second threshold. The first figure is a triangle formed by the three coordinates corresponding to the first plane, and the second figure is a triangle formed by the three coordinates corresponding to the second plane. Alternatively, the absolute value of the difference between the area of ​​the first graphic and the area of ​​the second graphic is less than or equal to the third threshold.

8. An electronic device, characterized in that, include: A processor and a memory, wherein the memory stores a computer program, and the processor invokes the computer program stored in the memory to perform the method as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on one or more processors, causes the method as described in any one of claims 1-5 to be performed.

Images