Registration method and related apparatus

Abstract

The embodiment of the application provides a registration method and related device, first determining N first reference coordinates of N markers accurately collected by an optical measurement device at a first reference moment, in the case that the number K of K first coordinates collected at a first moment later than the first reference moment is less than N, i.e. in the case that the markers are blocked, selecting the same number (i.e. M) of coordinates from the K first coordinates and the N first reference coordinates respectively for registration, in the case that the registration is successful, taking the obtained first registration matrix as the registration matrix between the K first coordinates and the N markers, so that the time spent in the registration process can be saved, the registration efficiency and accuracy can be improved, and the frequency of real-time updating of surgical navigation can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to registration methods and related apparatus. Background Technology

[0002] With the continuous development of computer science and technology, medical technology has also ushered in major breakthroughs, leading to the increasingly widespread application of minimally invasive surgery. Minimally invasive surgery relies on surgical navigation systems, which can be understood as accurately mapping the patient's preoperative or intraoperative computed tomography (CT) image data (i.e., CT images) to the patient's real-time anatomical structures on the operating table. During the surgeon's operation, the system tracks the surgical probe (also called the surgical needle) and updates and displays the position of the surgical needle on the patient's image in real time as a virtual probe.

[0003] In optical tracking-based surgical navigation systems, the coordinates of the surgical needle are obtained based on the coordinate system corresponding to the optical measurement equipment, while the CT image is obtained based on the coordinate system corresponding to the CT equipment. Because the two coordinate systems are different, misalignment will occur when they are displayed on the same interface. Currently, markers can be attached to the patient's body surface. The coordinates of these markers, determined by the CT equipment, are used as point cloud A, and the coordinates of these markers, determined by the optical measurement equipment, are used as point cloud B. A point cloud registration algorithm is used to register point clouds A and B. The resulting registration matrix transforms the surgical object in the CT coordinate system to the optical measurement coordinate system for display, achieving coordinate system unification and thus enabling surgical navigation. However, in real-world scenarios, the markers attached to the patient's body surface are often obscured, causing the registration process to be time-consuming and resulting in a lower frequency of real-time updates to the surgical navigation. Summary of the Invention

[0004] This application provides a registration method and related apparatus. The method provided by this application can save the time spent in the registration process and increase the frequency of real-time updates of surgical navigation.

[0005] In a first aspect, embodiments of this application provide a registration method, including:

[0006] N first reference coordinates are determined. The above N first reference coordinates are the coordinates of N markers collected by the optical measuring device at the first reference time. N is an integer greater than or equal to 3.

[0007] K first coordinates are obtained. The K first coordinates are determined based on the coordinates of the N markers collected by the optical measuring device at the first moment. The first moment is later than the first reference moment. K is an integer greater than or equal to 3.

[0008] When K is less than N, the M first reference coordinates and M first coordinates are registered based on the registration algorithm. The M first reference coordinates are selected from the N first reference coordinates, and the M first coordinates are selected from the K first coordinates. M is an integer greater than or equal to 3.

[0009] If the registration process is successful, the first registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates. The first registration matrix is ​​obtained through the registration process and is the registration matrix between the M first reference coordinates and the M first coordinates.

[0010] In one possible implementation, determining the N first reference coordinates includes:

[0011] In response to the first triggering operation, the above-mentioned N first reference coordinates are determined. The first triggering operation is used to indicate that the implementation object is in a first state. The above-mentioned N markers are pasted on the body surface of the implementation object.

[0012] The above process of obtaining K first coordinates includes:

[0013] In response to the second triggering operation, the aforementioned K first coordinates are obtained. The second triggering operation is used to indicate that the implementation object is in a second state, and the stability level corresponding to the first state is greater than the stability level corresponding to the second state.

[0014] In one possible implementation, before performing the registration process on the M first reference coordinates and the M first coordinates based on the registration algorithm, the method further includes:

[0015] Obtain the historical registration matrix, which is the registration matrix between the N first reference coordinates and the N second reference coordinates. The N second reference coordinates are the coordinates of the N markers collected by the optical measuring device at the second reference time. The second reference time is later than the first reference time and earlier than the first time. The time interval between the first reference time and the second reference time is less than or equal to the first time threshold.

[0016] The above-mentioned registration process based on the registration algorithm for M first reference coordinates and M first coordinates includes:

[0017] Based on the above historical registration matrix, the first error is obtained in registering the above K first coordinates to the above N first reference coordinates;

[0018] If the first error is greater than the first threshold, the above-mentioned registration process is performed on the above-mentioned M first reference coordinates and the above-mentioned M first coordinates based on the above-mentioned registration algorithm;

[0019] If the first error is less than the first threshold, the historical registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates.

[0020] In one possible implementation, the above method further includes:

[0021] If the registration process fails, the historical registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates.

[0022] In one possible implementation, the above-described registration process based on the registration algorithm for M first reference coordinates and M first coordinates includes:

[0023] Based on the historical registration matrix, the above M first coordinates are subjected to the first registration process to obtain M processed first coordinates. The first distance is less than the second distance. The first distance is the average of the first Euclidean distance between each of the above M processed first coordinates and the corresponding coordinates in the above N first reference coordinates. The second distance is the average of the second Euclidean distance between the above M first coordinates and the corresponding coordinates in the above N first reference coordinates.

[0024] Based on the above registration algorithm, a second registration process is performed on the above M first reference coordinates and the above M processed first coordinates.

[0025] In one possible implementation, obtaining the K first coordinates includes:

[0026] Obtain H original first coordinates, where the H original first coordinates are the coordinates of the N markers collected by the optical measuring device at the first moment, and H is an integer greater than K.

[0027] The above K first coordinates are determined from the above H original first coordinates. The angle between the first plane and the second plane is less than or equal to the second threshold. The above first plane is obtained based on three of the above K first coordinates, and the above second plane is obtained based on three of the above N first reference coordinates.

[0028] In one possible implementation, the absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to a third threshold, wherein the first figure is a triangle formed by three coordinates corresponding to the first plane, and the second figure is a triangle formed by three coordinates corresponding to the second plane.

[0029] Secondly, embodiments of this application provide a registration apparatus, including:

[0030] The determining unit is used to determine N first reference coordinates, wherein the N first reference coordinates are the coordinates of N markers collected by the optical measuring device at a reference time, and N is an integer greater than or equal to 3;

[0031] The acquisition unit is used to acquire K first coordinates, which are determined based on the coordinates of the N markers collected by the optical measuring device at a first moment. The first moment is later than the reference moment, and K is an integer greater than or equal to 3.

[0032] The registration unit is used to perform registration processing on M first reference coordinates and M first coordinates based on the registration algorithm. The M first reference coordinates are selected from the N first reference coordinates, and the M first coordinates are selected from the K first coordinates. M is an integer greater than or equal to 3.

[0033] The aforementioned determining unit is further configured to, when the result of the aforementioned registration process is successful registration, use the first registration matrix as the registration matrix between the aforementioned K first coordinates and the aforementioned N first reference coordinates, wherein the aforementioned first registration matrix is ​​obtained through the aforementioned registration process and is the registration matrix between the aforementioned M first reference coordinates and the aforementioned M first coordinates.

[0034] In one possible implementation, the registration device further includes a response unit, which responds to a first trigger operation and controls the determination unit to determine the N first reference coordinates. The first trigger operation is used to indicate that the implementation object is in a first state, and the surface of the implementation object is covered with the N markers.

[0035] The aforementioned response unit is further configured to respond to the second triggering operation and control the aforementioned acquisition unit to acquire the aforementioned K first coordinates. The aforementioned second triggering operation is configured to indicate that the aforementioned implementation object is in a second state, and the stability level corresponding to the aforementioned first state is greater than the stability level corresponding to the aforementioned second state.

[0036] In one possible implementation, the acquisition unit is further configured to acquire a historical registration matrix, wherein the historical registration matrix is ​​a registration matrix between the N first reference coordinates and the N second reference coordinates, wherein the N second reference coordinates are the coordinates of the N markers acquired by the optical measuring device at a second reference time, wherein the second reference time is later than the first reference time and earlier than the first time, and the time interval between the first reference time and the second reference time is less than or equal to a first time threshold.

[0037] The aforementioned acquisition unit is specifically used to acquire the first error of registering the aforementioned K first coordinates to the aforementioned N first reference coordinates based on the aforementioned historical registration matrix;

[0038] The registration unit is specifically used to perform the registration process on the M first reference coordinates and the M first coordinates based on the registration algorithm when the first error is greater than the first threshold.

[0039] The registration unit is specifically used to use the historical registration matrix as the registration matrix between the K first coordinates and the N first reference coordinates when the first error is less than the first threshold.

[0040] In one possible implementation, the determining unit is further configured to use the historical registration matrix as the registration matrix between the K first coordinates and the N first reference coordinates if the registration process results in a registration failure.

[0041] In one possible implementation, the registration unit is specifically used to perform a first registration process on the M first coordinates based on the historical registration matrix to obtain M processed first coordinates, where the first distance is less than the second distance. The first distance is the average of the first Euclidean distances between each of the M processed first coordinates and the corresponding coordinates in the N first reference coordinates, and the second distance is the average of the second Euclidean distances between the coordinates in the M first coordinates and the corresponding coordinates in the N first reference coordinates.

[0042] The aforementioned registration unit is specifically used to perform a second registration process on the aforementioned M first reference coordinates and the aforementioned M processed first coordinates based on the aforementioned registration algorithm.

[0043] In one possible implementation, the acquisition unit is specifically used to acquire H original first coordinates, wherein the H original first coordinates are the coordinates of the N markers collected by the optical measuring device at the first moment, and H is an integer greater than K.

[0044] The aforementioned determining unit is specifically used to determine the aforementioned K first coordinates from the aforementioned H original first coordinates, wherein the included angle between the first plane and the second plane is less than or equal to a second threshold, the aforementioned first plane is obtained based on three coordinates from the aforementioned K first coordinates, and the aforementioned second plane is obtained based on three coordinates from the aforementioned N first reference coordinates.

[0045] In one possible implementation, the absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to a third threshold, wherein the first figure is a triangle formed by three coordinates corresponding to the first plane, and the second figure is a triangle formed by three coordinates corresponding to the second plane.

[0046] Thirdly, embodiments of this application disclose an electronic device, including: a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program stored in the memory to perform a method as described in the first aspect or any possible implementation of the first aspect.

[0047] Fourthly, this application also provides another electronic device, including: a processor, a transmitting device, an input device, an output device, and a memory, wherein the memory is used to store computer program code, the computer program code including computer instructions, and when the processor executes the computer instructions, the electronic device performs the method as described in the first aspect or any possible implementation of the first aspect.

[0048] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on one or more processors, causes the method as described in the first aspect or any possible implementation thereof to be executed.

[0049] In a sixth aspect, embodiments of this application provide a computer program product including program instructions that, when executed by a processor, cause the processor to perform a method as described in the first aspect or any possible implementation thereof. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments or background art of this application, the accompanying drawings used in the embodiments or background art of this application will be briefly introduced below.

[0051] Figure 1 This is a schematic diagram of an optical measurement system 100 provided in an embodiment of this application;

[0052] Figure 2 This is a schematic diagram of a scenario provided by an embodiment of this application for registering the coordinate system corresponding to an optical measurement device with the coordinate system corresponding to a CT scanner;

[0053] Figure 3 This is a flowchart illustrating a registration method between the coordinate system of a CT scanner and the coordinate system of an optical measurement device, as provided in an embodiment of this application.

[0054] Figure 4 This is a schematic diagram illustrating the registration of the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device at time B, according to an embodiment of this application.

[0055] Figure 5 This is a flowchart illustrating a registration method provided in an embodiment of this application;

[0056] Figure 6 This is a schematic diagram illustrating a scenario where an optical measuring device generates artifact points during the acquisition of the coordinates of a marker, as provided in an embodiment of this application.

[0057] Figure 7 This is a schematic diagram illustrating how a normal vector is obtained from a plane formed by markers, according to an embodiment of this application.

[0058] Figure 8 This is a schematic diagram illustrating the determination of a historical registration matrix provided in an embodiment of this application;

[0059] Figure 9 This is a flowchart illustrating another registration method provided in an embodiment of this application;

[0060] Figure 10 This is a schematic diagram of the structure of a registration device provided in an embodiment of this application;

[0061] Figure 11 This is a schematic diagram of another registration device provided in an embodiment of this application. Detailed Implementation

[0062] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items. The terms “first” and “second,” etc., in the specification, claims, and drawings of this application are used to distinguish different objects and not to describe a particular order.

[0063] With the continuous development of computer science and technology, medical technology has also achieved significant breakthroughs, leading to the increasingly widespread application of minimally invasive surgery. Compared to open surgery, minimally invasive surgery is characterized by smaller incision areas, lower infection rates, faster patient recovery, and shorter hospital stays. However, because of the smaller incisions, surgeons cannot directly visualize the patient's internal organs and tissue structures, thus losing direct visual feedback from the real-time surgical process. Furthermore, the limited intraoperative field of view necessitates repeated review of preoperative images, making minimally invasive surgery reliant on surgical navigation systems.

[0064] In this embodiment, the surgical navigation system can be understood as using an optical measurement system to accurately correlate the anatomical structure reconstructed from the patient's preoperative CT image data (i.e., CT images) with the real-time anatomical structure of the patient on the operating table. During the surgery, the system tracks the surgical probe (also called the surgical needle) and updates its position on the patient's image in real time as a virtual probe. The surgical navigation system allows doctors to determine the position of the surgical needle relative to the patient's anatomical structure, making minimally invasive surgery faster, more precise, and safer. For example, in brain surgery, the surgical navigation system can display a "virtual human brain" of the patient on a monitor. During the surgery, the system displays the real-time position of the surgical needle in the doctor's hand, allowing the doctor to know whether the tumor edge has been reached, whether there is important tissue in front of the probe, etc., assisting the doctor in the surgery.

[0065] In this embodiment, an optical measurement device can be used to map surgical needles, the patient's body, and other objects in a real-world scenario onto a display screen in real time. This allows doctors to obtain more information through the display, thereby improving surgical precision and reducing the doctor's workload. For example, please refer to... Figure 1 , Figure 1 This is a schematic diagram of an optical measurement system 100 provided in an embodiment of this application. Figure 1 As shown, the optical measurement system 100 includes an optical measurement device 101, an electronic device 102, and a marker 103. Additionally, Figure 1 The part 104 in the document can be understood as the implementation object, and for ease of understanding, it will be referred to as implementation object 104 thereafter. In the implementation of this application, the implementation object can be understood as a patient (e.g., a patient who needs to undergo surgery), or, in some scenarios, it can also be understood as other objects or users that need to collect data through the aforementioned optical measurement system. For ease of understanding, the term "implementation object" will be used as the main subject in the following explanation. Figure 1 The part 105 in the document can be understood as the communication connection between the optical measuring device 101 and the electronic device 102. For ease of understanding, it will be referred to as communication connection 105. It should be understood that the above-mentioned communication connection 105 can be a wired connection or a wireless connection, and this application does not limit it in this regard.

[0066] In this embodiment, the marker 103 can be either a passive or an active marker. A passive marker has a reflective coating on its surface that reflects (rather than scatters) infrared (IR) light back to its source. An active marker includes an infrared receiver that emits infrared light when the receiver detects an infrared pulse. It is understood that the marker can be affixed to different locations in different application scenarios. For example, the marker can be affixed to an application tool, such as a surgical needle; and also, for example, the marker can be affixed to the surface of an object, such as... Figure 1 As shown, the surface of the embodiment 104 is covered with markers 103, which, exemplarily, include 12 markers.

[0067] In this embodiment, the optical measuring device 101 can be understood as a device for measuring the spatial position information of a marker. Optionally, the optical measuring device 101 may include a position sensor, which may include an infrared light-emitting diode and an IR sensor, etc. For ease of understanding, ... Figure 1 Taking marker 1031 as an example, for instance, marker 1031 is a passive marker. Figure 1 As shown, the optical measuring device 101 generates infrared light through an infrared light-emitting diode and illuminates the marker 1031. The reflective coating on the surface of the marker 1031 reflects the infrared light back to the IR sensor of the optical measuring device 101, and then transmits the light through a reflective coating. Figure 1 The intersection of the light rays shown (i.e., the arrows shown in dashed lines) enables the triangulation of the three-dimensional spatial coordinates of marker 1031.

[0068] It is understood that a communication connection 105 exists between the electronic device 102 and the optical measuring device 101. After the optical measuring device 101 acquires the three-dimensional spatial coordinates of the marker, the electronic device 102 can acquire the coordinate data of the marker through the aforementioned communication connection 105. Optionally, the electronic device 102 includes a display. Therefore, after the electronic device 102 acquires the three-dimensional spatial coordinates of the marker, the marker can be displayed on the display with the coordinate system of the optical measuring device 101 as a reference.

[0069] Understandably, during surgery, the surgical needle is positioned using markers attached to the surface and optical measuring devices. After acquiring the needle's position information through the optical measuring devices, the electronic equipment displays the needle using the coordinate system corresponding to the optical measuring devices. However, on the other hand, after acquiring CT images through a CT scanner, the electronic equipment displays the tissue structures (skin, bone, lungs, etc.) reconstructed from the CT images using the coordinate system corresponding to the CT scanner. Because the coordinate systems of the optical measuring devices and the CT scanners are different, misalignment will occur when displayed on the same interface. For example, the surgical needle may actually be 5cm from the patient's body surface, but the monitor may display it as if the needle has penetrated the patient's body or is elsewhere off-center. Therefore, registration (or unification) between the coordinate systems of the optical measuring devices and the CT scanners is a crucial step.

[0070] For example, point cloud registration can be used to register coordinates between different coordinate systems. In this embodiment, a point cloud can be understood as a set of multiple points, and point cloud registration can be understood as solving the rotation and translation matrix between two point clouds to transform the source point cloud to the same coordinate system as the target point cloud. The rotation and translation matrix can be a rigid transform or an Euclidean transform. For example, iterative closest point (ICP) is an important algorithm in point cloud registration, and the ICP algorithm can be expressed by formula (1):

[0071] P t =R·P s +T (1)

[0072] Among them, P t This can be understood as the target point cloud, P s This can be understood as the source point cloud. The ICP algorithm process involves solving for the rotation matrix R and the translation matrix T, such that the point cloud P... s After the above transformations (rotation and translation) and the point cloud P t The results are obtained by overlapping and iterating continuously to minimize the mean square error.

[0073] To better understand this scheme, the ICP algorithm described above will be briefly introduced below. For example, assuming a transformation point set X and a reference point set Y, registration can be performed through the following steps:

[0074] Step a: Select three points from the set of transformed points X, denoted as points X1, X2, and X3;

[0075] Step b: Find three points corresponding to points X1, X2, and X3 from the reference point set Y through exhaustive search, and denote them as points Y1, Y2, and Y3. Translate the center points of points X1, X2, and X3 to the center points of points Y1, Y2, and Y3 to obtain the translation matrix T1 between points X1, X2, and X3 and points Y1, Y2, and Y3. Then, use the singular value decomposition (SVD) method to calculate the rotation matrix R1 between points X1, X2, and X3 and points Y1, Y2, and Y3. Superimpose T1 and R1 to obtain the transformation matrix M1.

[0076] Step c: Multiply the transformation matrix M1 by the point set X to obtain the transformed point set X' and calculate the registration error, such as the root mean square (RMS) error. If the RMS error is greater than a certain threshold, it is considered that points X1, X2, and X3 are not corresponding point sets with points Y1, Y2, and Y3. Return to step 2 to continue exhaustively searching for corresponding points.

[0077] Step d: After finding the corresponding point set, sort the points in the transformed point set X' according to their distance from the points in the reference point set Y. Also, the points in the transformed point set X' that are closest to the points in the reference point set Y have the same index. Then record the order of change of the points in the transformed point set X'.

[0078] Step e: Calculate the rotation matrix R2 based on the transformation point set X' and the reference point set Y, and multiply the rotation matrix R2 by the transformation point set X' to obtain the transformation point set X”.

[0079] Step f: If the average distance between the transformed point set X” and the reference point set Y is less than the reference threshold, stop the iterative calculation. If the average distance between the transformed point set X” and the reference point set Y is greater than the reference threshold, repeat steps d and e until the number of iterations exceeds the reference number. If the number of iterations exceeds the reference number, repeat steps a-e. During the re-execution of steps a-e, the order of the transformed point set X is adjusted using the change order, so that the order of the corresponding points in the transformed point set X” is the same as the order of the points in the transformed point set X”. The transformation matrix is ​​then calculated using the transformed point set X” with the adjusted order.

[0080] In this embodiment, the registration refers to the registration between the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device. To better understand the registration process of this scheme, the following will be combined with... Figure 2 and Figure 3 The specific process of registration is described. Figure 2 This is a schematic diagram of a scenario provided by an embodiment of this application for registering the coordinate system corresponding to an optical measurement device with the coordinate system corresponding to a CT scanner; Figure 3 This is a flowchart illustrating a registration method between the coordinate system of a CT scanner and the coordinate system of an optical measurement device, as provided in an embodiment of this application. It should be understood that... Figure 3 The registration method shown can be performed by a registration device.

[0081] like Figure 2 As shown in section 201, multiple markers are affixed to the surface of the implementation object, among which, Figure 2 Five markers are illustrated exemplarily. It should be understood that the embodiments of this application achieve registration between the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device based on the coordinates of the aforementioned markers under the CT equipment and the coordinates of the aforementioned markers under the optical measurement device. Therefore, the registration device processes data acquired by the CT equipment and the optical measurement device respectively. The scenario for processing the data acquired by the CT equipment is as follows: Figure 2 As shown in sections 2011 and 2012, the registration device's processing of data acquired by the CT equipment includes... Figure 3 Steps 301-304 in the diagram. The scenario for processing data acquired by the optical measurement equipment is as follows: Figure 2 As shown in sections 2013 and 2014, the processing procedure of the registration device for the data acquired by the optical measurement equipment includes... Figure 3 Steps 305-306 in the example. It should be noted that the numbering before each step in this embodiment is for ease of understanding of the scheme and does not limit the execution order of each step.

[0082] Furthermore, it should be understood that in real-world scenarios, coordinate system registration is a real-time process. Taking a surgical procedure as an example, the two coordinate systems mentioned above need to be registered throughout the entire surgical process to effectively guide the surgical operation. Therefore, for ease of understanding, as... Figure 3 As shown, the above registration method is divided into a first stage and a second stage. The first stage can be understood as registration before surgery, and the second stage can be understood as registration during surgery.

[0083] Next, combine Figure 3 The first stage described above describes the data processing process of the registration device on the CT equipment. This processing process includes:

[0084] 301: Acquire multiple CT images captured by the CT equipment at time A.

[0085] The scenario corresponding to this step can be found in [reference]. Figure 2 In part 2011, the subject is pushed into the CT equipment for CT scanning, and the CT equipment acquires the aforementioned multiple CT images at the aforementioned time A.

[0086] In this step, time A can be understood as the first reference time in the embodiments of this application. For example, the registration device can establish a communication connection with the CT equipment and obtain multiple CT images of the implementation object at time A through the communication connection.

[0087] 302: Three-dimensional reconstruction based on these multiple CT images.

[0088] In this step, the registration device segments skin tissue and bone contours from the multiple CT images to achieve three-dimensional reconstruction of the internal organs and tissues of the target object. Optionally, the registration device includes a display, so the registration device can display the internal structure of the target object in the form of a three-dimensional view on the display.

[0089] 303: Determine the coordinates of multiple markers from the multiple CT images to obtain coordinate A.

[0090] The scenario corresponding to this step can be found in [reference]. Figure 2 In the 2012 section, it should be understood that multiple markers were affixed to the surface of the subject's body, and the multiple CT images obtained by CT scanning of the subject included these multiple markers, such as... Figure 2 In the 2012 section, the CT images included five markers pasted on the subject's body surface.

[0091] In this step, the registration device segments the multiple markers from the multiple CT images to obtain the coordinates A of each marker in the coordinate system corresponding to the CT equipment. For example, as shown... Figure 2 In the 2012 part, the coordinates corresponding to the 5 markers in the CT image can be understood as the coordinates A mentioned above.

[0092] It is understood that when performing three-dimensional reconstruction based on the above multiple CT images in step 302, multiple markers in the above multiple CT images can be displayed in the reconstructed three-dimensional view. These multiple markers are obtained by segmentation and reconstruction based on the CT sequence.

[0093] 304: Take coordinate A as point cloud A.

[0094] In this step, it should be understood that the aforementioned coordinate A refers to the coordinates of each of the multiple markers in the coordinate system corresponding to the CT equipment; that is, the aforementioned coordinate A can be understood as a collective term for multiple coordinates. For example, as shown... Figure 2 In the 2012 part, the coordinates of the five markers in the CT image can be understood as the point cloud A mentioned above.

[0095] The following describes the data processing process of the registration device on the CT equipment in the first stage described above. The processing process includes:

[0096] 305: Obtain the coordinates of the multiple markers collected by the optical measuring device at time A, and obtain coordinates B.

[0097] The scenario corresponding to this step can be found in [reference]. Figure 2 In section 2013, the object being measured is placed within the detection range of the optical measuring equipment. In this step, the optical measuring equipment acquires the coordinates of multiple markers attached to the surface of the object. Specific acquisition methods can be found in the preceding text. Figure 1 The relevant descriptions will not be repeated here. For example, as shown... Figure 2 In the 2014 section, the optical measuring equipment collected the coordinates of five markers pasted on the surface of the object being measured. The coordinates of these five markers can be understood as coordinates B mentioned above.

[0098] 306: Use coordinate B as point cloud B.

[0099] In this step, it should be understood that the aforementioned coordinate B refers to the coordinates of each of the multiple markers in the coordinate system corresponding to the optical measuring device; that is, the aforementioned coordinate B can be understood as a collective term for multiple coordinates. For example, as shown... Figure 2 In the 2014 section, the coordinates of the five markers collected by the optical measurement equipment can be understood as the point cloud B mentioned above.

[0100] After determining the point cloud A and the point cloud B, the registration device performs step 307: registering the point cloud A and the point cloud B based on the registration algorithm to obtain the registration matrix M0.

[0101] It should be understood that the coordinates in point cloud A are obtained based on the coordinate system corresponding to the CT equipment, while the coordinates in point cloud B are obtained based on the coordinate system corresponding to the optical measurement equipment. Registration between the two different coordinate systems can be performed using a registration algorithm to obtain the registration matrix M0. For example, as shown... Figure 2 As shown, the ICP algorithm can be used to register point cloud A and point cloud B.

[0102] It should be understood that a point cloud is a collection of multiple points. In the embodiments of this application, each marker can be understood as a point, and the multiple markers can be combined to form a point cloud. However, in the process of processing each point (that is, each marker), the registration device converts the coordinates of each point into a matrix for processing. In other words, registration can be understood as the registration between two point clouds. Registration is achieved by converting the coordinates of the point cloud into matrix form and performing calculations and transformations. A point is an entity, and coordinates are the spatial position expression of a point. Therefore, it can also be understood as the registration between multiple coordinates, such as the registration between coordinate A and coordinate B, and the registration between coordinate A and coordinate C.

[0103] After obtaining the above registration matrix M0, it can also be understood that the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device have been registered at time A. That is, in addition to the above-mentioned multiple markers, other objects in the coordinate system corresponding to the CT device, such as skin, bones and lung organs obtained from the above-mentioned multiple CT images, can be transformed into the coordinate system corresponding to the optical measurement device through the above-mentioned registration matrix M0.

[0104] In this embodiment, steps 301-307 in the first stage can be understood as registration between coordinate systems at time A. In subsequent processes, the breathing and other limb movements of the subject will cause changes in the coordinates of the multiple markers acquired by the optical measurement device. Therefore, after time A, the registration device needs to reacquire the coordinates of the multiple markers to achieve real-time registration between coordinate systems. For ease of understanding, the registration after time A is referred to as the second-stage registration. It should be understood that the scenario of the optical measurement device acquiring markers during the second-stage registration process can still be referred to... Figure 2 The 2013 and 2014 sections. For example... Figure 3 As shown, the second-stage registration process includes:

[0105] 308: Obtain the coordinates of the multiple markers collected by the optical measuring device at time B, and obtain coordinate C.

[0106] In this step, the aforementioned time B is later than the aforementioned time A, and the aforementioned time B can be understood as the first time in the embodiment of this application.

[0107] 309: Use the coordinates C as the point cloud C.

[0108] In this step, it should be understood that the coordinate C mentioned above refers to the coordinates of each of the multiple markers in the coordinate system corresponding to the optical measuring device. That is, the coordinate C mentioned above can be understood as a collective term for multiple coordinates.

[0109] 310: Based on the registration algorithm, point cloud B and point cloud C are registered to obtain the registration matrix M. h .

[0110] In this step, the coordinates B in point cloud B and C in point cloud C are coordinates acquired by the optical measuring device at different times. That is, coordinate B is the coordinate acquired at time A, and coordinate C is the coordinate acquired at time B. Therefore, the registration matrix M... h This can be understood as registering the positions and attitudes of the aforementioned markers at time B to the positions and attitudes at time A using a registration matrix. The registration algorithm in this step can also be the ICP algorithm described above.

[0111] 311: Based on M0 and Mh The registration matrix M between coordinate C and coordinate A at time A is obtained.

[0112] 312: Based on the registration matrix M, register the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device.

[0113] To facilitate understanding of the above steps, please refer to the example provided. Figure 4 , Figure 4 This is a schematic diagram illustrating the registration of the coordinate system corresponding to the CT device and the coordinate system corresponding to the optical measurement device at time B, as provided in an embodiment of this application.

[0114] like Figure 4 As shown, the registration matrix M0 is the registration matrix of the two coordinate systems obtained through coordinates A and B at time A. h The registration matrix for the position and orientation of the marker at times A and B, obtained using coordinates B and C, is thus calculated by performing the operation M0×M. h The obtained matrix M can be used as the registration matrix of the two coordinate systems at time B. It should be understood that the two coordinate systems mentioned above refer to the coordinate system corresponding to the CT equipment and the coordinate system corresponding to the optical measurement equipment.

[0115] It should be understood that, through the above-mentioned registration matrix M, the object in the coordinate system corresponding to the optical measurement device can be registered to the coordinate system corresponding to the CT device. In some embodiments, by performing an inverse operation on the above-mentioned registration matrix M, the object in the coordinate system corresponding to the CT device can be registered to the coordinate system corresponding to the optical measurement device.

[0116] In real-world scenarios, in the above Figure 3 In the second stage, during the real-time acquisition of the coordinates of the markers by the optical measuring equipment, the markers attached to the subject's body surface may be obscured. For example, a doctor or nurse may inadvertently obscure part of the marker during surgery, or the subject's body movement may cause part of the marker to move out of the detection range of the optical measuring equipment. Therefore, in the above... Figure 3 In the second stage, the number of coordinates of the markers collected in real time by the optical measurement equipment may be less than the number of markers actually pasted on the surface of the object. As a result, the number of coordinates in point cloud A and point cloud B will be unequal, leading to a long registration time and low registration efficiency between the two coordinate systems.

[0117] To address the aforementioned problems, this application provides a registration method and related apparatus, which can save time spent in the registration process. It is understood that the registration method provided in this application can be executed by a registration device, which can be any electronic device capable of executing the technical solutions disclosed in the method embodiments of this application. For example, the registration device can be a computer, tablet computer, desktop computer, etc., and this application does not limit it in this regard. It should also be understood that the method embodiments in this application can also be implemented by a processor executing computer program code.

[0118] For example, please refer to Figure 5 , Figure 5 This is a flowchart illustrating a registration method provided in an embodiment of this application, as shown below. Figure 5 As shown, the above method includes:

[0119] 501: Determine N first reference coordinates, which are the coordinates of N markers collected by the optical measuring device at the first reference time, where N is an integer greater than or equal to 3.

[0120] In this application embodiment, the aforementioned marker can also be referred to as an optical marker ball, a reflective marker, a reflective marker point (mark point), etc., and this application does not impose any limitation on this. Furthermore, the aforementioned marker can be an active marker or a passive marker; for an explanation of active and passive markers, please refer to the preceding text. Figure 1 The relevant descriptions will not be repeated here. In this embodiment, the above-mentioned N markers can be understood as real markers existing in the actual scene. Depending on the different scenes, the above-mentioned N markers can be pasted in different locations. For example, the above-mentioned N markers can be pasted on the body surface of the subject, such as the chest, abdomen, etc.

[0121] In this embodiment, the optical measuring device can use infrared light to acquire the coordinates of the marker; the specific acquisition method can be found in the preceding text. Figure 1 The relevant descriptions will not be repeated here. It should be understood that the coordinates of the markers collected by the optical measuring equipment are three-dimensional spatial coordinates. That is to say, the above-mentioned reference coordinates, as well as the first coordinate in step 502 below, are all three-dimensional spatial coordinates.

[0122] In this embodiment, the first reference time can be understood as the time when the optical measuring device accurately acquires the coordinates of each of the N markers. That is, when the optical measuring device acquires the coordinates of the N markers at the first reference time, it can accurately acquire N coordinates, and each of the N coordinates corresponds one-to-one with each of the N markers. For example, when the object is in a stable state (e.g., holding its breath or breathing steadily), the optical measuring device can be adjusted to accurately acquire the coordinates of each marker. For example, the registration device can establish a communication connection with the optical measuring device to obtain the coordinates of each of the N markers acquired by the optical measuring device at the reference time, and then use the coordinates of each marker as the N first reference coordinates.

[0123] 502: Obtain K first coordinates, which are determined based on the coordinates of the N markers collected by the optical measuring device at the first moment. The first moment is later than the first reference moment, and K is an integer greater than or equal to 3.

[0124] In this step, the first coordinates of the aforementioned N markers at the first moment can be acquired by the optical measuring device in step 501. It is understood that in practical applications, after the markers are attached to the surface of the subject, the optical measuring device will continuously acquire the coordinates of the markers over a period of time (e.g., throughout the entire surgical procedure), as described above. Figure 3 The second stage of the method shown. In the above scenario, the first moment can be understood as any moment during the surgical procedure. As mentioned above. Figure 3 As can be understood from the relevant description, deviations may occur during the process of the optical measuring device acquiring the coordinates of the markers, resulting in the optical measuring device obtaining K coordinates when acquiring the coordinates of the N markers at the first moment, i.e., the aforementioned K first coordinates. In this embodiment, K is an integer greater than or equal to 3, and the explanation for this can be found in the description in step 503, which will not be repeated here.

[0125] Exemplarily, the registration device can establish a communication connection with an optical measurement device to obtain K first coordinates of the N markers at a first moment. In some embodiments, the K first coordinates can be understood as the coordinates of the N markers acquired in real time by the optical measurement device. In some embodiments, the K first coordinates can be understood as coordinates obtained after the following processing; for ease of understanding, the following processing can be understood as an artifact point processing process. Exemplarily, in some embodiments, step 502 includes:

[0126] 5021. Obtain H original first coordinates, where the H original first coordinates are the coordinates of the N markers collected by the optical measuring device at the first moment, and H is an integer greater than K;

[0127] It should be understood that in some scenarios, the surgical environment may contain reflective objects, causing the markers detected by the optical measuring equipment to have flashing points. In other scenarios, artifacts may also occur when the optical measuring equipment calculates the coordinates of the markers using infrared light, resulting in the optical measuring equipment acquiring more coordinates than N when collecting the coordinates of the aforementioned N markers; that is, the number of acquired coordinates is greater than the number of markers actually attached to the surface of the subject. For an example, please refer to [link to relevant documentation]. Figure 6 , Figure 6 This is a schematic diagram of a scenario where an optical measuring device generates artifact points during the process of acquiring the coordinates of a marker, as provided in an embodiment of this application.

[0128] like Figure 6 As shown, Figure 6 Markers 602 and 603 can be understood as real-world markers; that is, the optical measuring device 601 needs to acquire the three-dimensional spatial coordinates of markers 602 and 603. It can be understood that when the optical measuring device 601 acquires the coordinates of the markers, it calculates the line between the marker and each IR sensor. For example... Figure 6 As shown, the optical measurement device 601 calculates the lines between marker 602 and IR sensor 6011, the lines between marker 602 and IR sensor 6012, the lines between marker 603 and IR sensor 6011, and the lines between marker 603 and IR sensor 6012, ultimately as follows: Figure 6 As shown by the dashed line in the image.

[0129] When the lines between the aforementioned markers and the IR sensor intersect or nearly intersect, the optical measurement device 601 calculates the distance between the intersecting or nearly intersecting lines. If the distance is less than a certain threshold, the optical measurement device 601 considers the intersection of the two lines as a marker as well. Figure 6 The white spherical marker in the middle. However, it can be understood that, as Figure 6 The white spherical markers in the image are not real markers, but rather artifacts.

[0130] It should be understood that when only the markers are obscured, the number of coordinates K collected by the optical measuring device is less than the number of markers N. When the markers are obscured and artifacts exist, the number of coordinates K collected by the optical measuring device may be less than, equal to, or greater than the number of markers N. In this step, the aforementioned H original first coordinates can be understood as the coordinates of the N markers detected by the optical measuring device before artifact processing. After artifact processing, the aforementioned K first coordinates are obtained from the aforementioned H original first coordinates. That is, each of the aforementioned K first coordinates can be understood as the coordinate of a certain marker pasted on the surface of the object. Therefore, in this embodiment, the aforementioned H is greater than the aforementioned K. The specific processing procedure can be referred to the relevant description in step 2.

[0131] 5022. Determine the K first coordinates from the H original first coordinates. The angle between the first plane and the second plane is less than or equal to the second threshold. The first plane is obtained from three of the K first coordinates, and the second plane is obtained from three of the N first reference coordinates.

[0132] This step can be understood as the registration device determining the actual K first coordinates from the aforementioned H original first coordinates. In this step, three of the aforementioned K first coordinates form a first plane, and any three of the aforementioned N first reference coordinates form a second plane. The angle between the first plane and the second plane is less than or equal to a second threshold.

[0133] In this embodiment, the angle between planes can be determined by the normal vector of the planes, and the range of the angle between two planes is [0, π / 2]. For example, after obtaining the H original first coordinates, the registration device can form a candidate plane from any three of the H original first coordinates, and use the plane formed by the three coordinates of the N first reference coordinates as the reference plane. It is understood that since N is greater than or equal to 3, when N equals 3, the reference plane is a single plane; when N is greater than 3, the reference plane should be understood as a set of multiple planes. For example, when N equals 5, the optical measuring device can acquire 5 reference coordinates through infrared light. Based on these 5 reference coordinates, 10 (i.e., C53 = 10) planes can be obtained, and these 10 planes can be understood as the reference plane. In this embodiment, the second plane can be understood as one of the reference planes.

[0134] Then, the registration device can calculate the angle A between the normal vector A of the candidate plane and the normal vector B of the second plane. If the angle A is less than or equal to the second threshold, the three coordinates corresponding to the candidate plane are used as the coordinates among the K first coordinates. In this case, the candidate plane can be understood as the first plane. It should be understood that the first threshold can be adjusted according to the actual situation. For example, the first threshold can be 2°, 3°, 5°, etc., and this application does not limit it.

[0135] It should be understood that since the aforementioned second plane can be interpreted as one of the aforementioned reference planes, that is, the candidate plane formed by three of the aforementioned H original first coordinates only needs to have an angle between it and one of the aforementioned reference planes that is less than or equal to the aforementioned second threshold. Since the angle between the plane formed by the three of the H original first coordinates actually acquired at the first moment and the plane formed by the N first reference coordinates acquired at the reference moment is less than or equal to the aforementioned second threshold, it can be considered that the structural features corresponding to the three coordinates acquired at the first moment are very similar to the structural features corresponding to the three coordinates at the reference moment. Therefore, it can be considered that the three coordinates acquired at the first moment are the coordinates of the actual marker.

[0136] It is understandable that the aforementioned N markers are pasted on the surface of the object. Since the surface of the object is not a smooth plane, but can be understood as a curved surface with undulations, any three coordinates of the N reference coordinates of the aforementioned N markers acquired by the optical measuring device can form a plane, and the normal vector corresponding to the plane formed by different three coordinates can point in different directions. For example, please refer to... Figure 7 , Figure 7 This is a schematic diagram illustrating how a normal vector is obtained from a plane formed by markers, as provided in an embodiment of this application. Figure 7 As shown, Figure 7 Five markers are illustrated, namely marker A, marker B, marker C, marker D, and marker E. Figure 7 As shown, any three markers can form a plane. For example, normal vector A can be understood as the normal vector corresponding to the plane formed by markers A, D, and E; normal vector B can be understood as the normal vector corresponding to the plane formed by markers B, C, and E; and normal vector C can be understood as the normal vector corresponding to the plane formed by markers C, D, and E. Figure 7 As shown, normal vector A, normal vector B, and normal vector C point in different directions.

[0137] In this step, if the angle between the first plane and the second plane is less than or equal to the third threshold, it can be considered that the structural features of the three coordinates corresponding to the first plane are almost the same as those of the three coordinates at the first reference time. Since the coordinates collected at the first reference time are real coordinates, it can be considered that the coordinates corresponding to the third plane are real coordinates. In other words, the artifact point coordinates can be removed from the collected coordinates by means of the above method.

[0138] In this embodiment, after acquiring H original first coordinates at the first moment, artifact points are eliminated by the angle between the plane formed by three of the H original first coordinates and the plane formed by N first reference coordinates acquired at the reference moment. From this, the true K first coordinates are determined for subsequent processing, which can further improve the efficiency and accuracy of registration.

[0139] In some embodiments, the absolute value of the difference between the perimeter of the first graphic and the perimeter of the second graphic is less than or equal to a third threshold, the first graphic is a triangle formed by three coordinates corresponding to the first plane, and the second graphic is a triangle formed by three coordinates corresponding to the second plane.

[0140] In this embodiment, the coordinates corresponding to the plane (including the first plane and the second plane mentioned above) can be understood as the three coordinates that make up the plane. For example, coordinates A, B, and C make up plane A, and the coordinates corresponding to plane A can be understood as coordinates A, B, and C. In this embodiment, the third threshold can be set according to the actual situation. For example, the third threshold can be 3 mm, 4 mm, etc.

[0141] Optionally, the absolute value of the difference between the area of ​​the first graphic and the area of ​​the second graphic is less than or equal to a fourth threshold. The fourth threshold can be set according to actual conditions, and can be 3 square millimeters, 4 square millimeters, etc., which is not limited in this application.

[0142] In this embodiment, after forming a plane from the real-time collected coordinates and using the angle between this plane and the reference plane as a standard to determine the coordinates for registration, the perimeter or area of ​​the triangle formed by the coordinates corresponding to the plane is further used to determine the coordinates for registration. This can further effectively eliminate artifact points, and then use the coordinates corresponding to the real existing markers for registration, saving registration time and improving the efficiency and accuracy of registration.

[0143] For example, in one possible implementation, step 5022 above includes:

[0144] 1. From the H original first coordinates, three original first coordinates are determined, and the angle between the plane corresponding to the three original first coordinates and the second plane is less than or equal to the third threshold.

[0145] In this step, the registration device can randomly select three candidate coordinates from H original first coordinates, calculate the angle between the normal vector of the plane corresponding to the three candidate coordinates and the normal vector corresponding to the second plane, and if the angle is less than or equal to the third threshold, the three candidate coordinates are used as the three original first coordinates. If the angle is greater than the third threshold, the registration device reselects three candidate coordinates until the selected candidate coordinates meet the above conditions.

[0146] In this step, the angle between the plane corresponding to the three original first coordinates and the second plane is less than or equal to the third threshold. Therefore, the three original first coordinates can be understood as coordinates among the K first coordinates.

[0147] 2. Combine the remaining coordinates with two of the three original first coordinates to form a third plane. The remaining coordinates are any coordinates other than the three original first coordinates among the H original first coordinates.

[0148] In this step, the remaining coordinates can be combined with any two of the three original first coordinates to form the third plane. For example, the three original first coordinates are represented by coordinates A1, A2, and A3, respectively, and the remaining coordinates are represented by A... i Therefore, the aforementioned third plane can be A1, coordinates A2 and A... i The plane formed can also be coordinates A2, A3, and A. i The plane formed can also be coordinates A1, A3, and A. i The plane that forms the composition.

[0149] 3. If the angle between the third plane and the second plane is less than or equal to the third threshold, the coordinates corresponding to the third plane shall be used as the coordinates among the K first coordinates.

[0150] In this step, if the angle between the third plane and the second plane is less than or equal to the third threshold, it can be considered that the structural features of the three coordinates corresponding to the third plane are almost the same as those of the three coordinates at the first reference time. Since the coordinates collected at the first reference time are real coordinates, it can be considered that the coordinates corresponding to the third plane are real coordinates.

[0151] In this embodiment, the remaining coordinates are any coordinates other than the three first coordinates among the H original first coordinates. It can be understood that all coordinates other than the three first coordinates among the H original first coordinates can be determined in the above manner. The registration device can process the coordinates other than the three first coordinates among the H original first coordinates respectively using the above steps 4 and 5 until all the H original first coordinates have been processed.

[0152] In this embodiment, three original first coordinates are determined from the above H original first coordinates. The angle between the plane corresponding to the three original first coordinates and the second plane is less than or equal to the third threshold. Then, based on the three original first coordinates and each of the remaining coordinates, other coordinates for registration are determined. This can improve the efficiency of the registration device in determining the coordinates for registration, thereby improving the efficiency of registration.

[0153] It should be noted that in the embodiments of this application, the steps are numbered for better understanding of the solution, but the numbering before the steps does not limit the execution order of the steps. For example, steps 1-3 in the preceding text, and steps 4-7 in the following text, etc.

[0154] In some embodiments, Figure 5 In the method shown, step 501 includes:

[0155] 5011. In response to the first triggering operation, determine the above-mentioned N first reference coordinates, the above-mentioned first triggering operation is used to indicate that the implementation object is in a first state, and the above-mentioned N markers are pasted on the body surface of the implementation object;

[0156] Step 502 includes:

[0157] 5021. In response to the second triggering operation, the above-mentioned K first coordinates are obtained. The second triggering operation is used to indicate that the above-mentioned implementation object is in a second state, and the stability degree corresponding to the first state is greater than the stability degree corresponding to the second state.

[0158] In this embodiment, the first triggering operation can be understood as a user operation (e.g., a user operation by a doctor or other operator). For example, the registration device may include an input component, such as a keyboard, mouse, touchscreen, or touchpad, which can be used to implement the triggering operation. For example, the triggering operation can be implemented by clicking a software button with a mouse or by clicking a software button on a touchscreen.

[0159] In this embodiment, after triggering the first trigger operation, the registration device determines the N first reference coordinates. This can be understood as follows: in response to the first trigger operation, the registration device acquires coordinate data from other electronic devices and determines the N first reference coordinates based on the acquired coordinate data. Alternatively, in response to the first trigger operation, the registration device uses the acquired coordinates as the N first reference coordinates according to the instruction of the first trigger operation. For example, in response to a click operation on a first button (e.g., a "reference data acquisition" button), the registration device acquires coordinate data from an optical measurement device and uses the acquired coordinate data as the N first reference coordinates. In the above case, the click operation on the first button can be understood as the first trigger operation.

[0160] Similarly, the second triggering operation described above can be understood as a user operation (e.g., a user operation by a doctor or other operator). For example, the registration device may include an input component, such as a keyboard, mouse, touchscreen, or touchpad, which can be used to implement the triggering operation. For example, the triggering operation can be implemented by clicking a software button with a mouse, or by clicking a software button on a touchscreen.

[0161] In this embodiment, after the second triggering operation is triggered, the registration device acquires K first coordinates. For example, in response to a click on a second button (e.g., a "Reference Data Acquisition Complete" button, a "Surgery Start" button, etc.), the registration device acquires coordinate data from the optical measurement device and uses this coordinate data as the K first coordinates of the N markers at the first moment. In this case, the click on the second button can be understood as the second triggering operation.

[0162] In this embodiment, the first triggering operation is used to indicate that the implementation object is in a first state, and the second triggering operation is used to indicate that the implementation object is in a second state. The stability level corresponding to the first state is greater than the stability level corresponding to the second state. It can be understood that the N first reference coordinates are obtained based on the coordinate data of the N markers at the first reference time, and the K first coordinates are the coordinate data of the N markers at the first time. This embodiment uses the data collected by the implementation object in the first state with a higher stability level as reference data, which allows for more accurate subsequent registration.

[0163] It is understandable that the aforementioned N markers are affixed to the surface of the subject's body. The subject's breathing and other limb movements will affect the optical measuring device's coordinate measurement of the markers. Regarding the stability corresponding to the two states, it can be understood that the subject's limb movement amplitude in the first state is less than that in the second state. For example, the subject's breath-holding state can be considered the first state, and the free breathing state following the first state can be considered the second state.

[0164] 503: When K is less than N, the M first reference coordinates and M first coordinates are registered based on the registration algorithm. The M first reference coordinates are selected from the N first reference coordinates, and the M first coordinates are selected from the K first coordinates. M is an integer greater than or equal to 3.

[0165] In this embodiment, the registration algorithm used for registration can be the aforementioned ICP algorithm. The registration process can be understood as a registration process based on the ICP algorithm for M first reference coordinates and M first coordinates. In this step, the registration device selects the same number (i.e., M) of coordinates from the aforementioned N first reference coordinates and the aforementioned K first coordinates for registration. It can be understood that since two points form a line segment, and a line segment can rotate, matching two points can only limit two degrees of freedom. Therefore, when using the registration algorithm for three-dimensional registration, three or more coordinates are selected for registration, i.e., M is an integer greater than or equal to 3. Since the aforementioned M first coordinates are selected from the aforementioned K first coordinates, K is a number greater than or equal to 3.

[0166] Understandably, when the markers pasted on the object are obscured, the number of coordinates detected by the optical measuring device is less than N. The registration device determines the relationship between the number of first coordinates K and the number of reference coordinates N. If K is less than N, that is, if it is determined that the number of coordinates acquired by the optical measuring device in real time is less than the actual number of markers, the device performs registration processing on the M first reference coordinates and the M first coordinates based on the registration algorithm.

[0167] Optionally, when K equals N, the registration matrix between the K first coordinates and the N first reference coordinates is obtained based on the registration algorithm. That is, when it is determined that K equals N, i.e., the number of coordinates acquired in real-time by the optical measuring device equals the actual number of markers, and the markers are not obstructed, the K first coordinates and the N first reference coordinates are registered based on the registration algorithm. In this case, the number of coordinates in both the K first coordinates and the N first reference coordinates is equal, and direct registration can be performed using the registration algorithm.

[0168] 504: If the registration process is successful, the first registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates. The first registration matrix is ​​obtained through the registration process and is the registration matrix between the M first reference coordinates and the M first coordinates.

[0169] It is understandable that when K or N is greater than 3, the registration device can select M first reference coordinates from N first reference coordinates and M first coordinates from K first coordinates for registration multiple times. For example, when N equals 5, K equals 4, and M equals 3, the registration device can perform 10 (i.e., C5) 3 =10) times to select 3 different reference coordinates from the above 5 reference coordinates, and 4 (i.e. C43=4) times to select 3 different first coordinates from the above 4 first coordinates.

[0170] It should be understood that each time the M first reference coordinates are selected and the M first coordinates are used for registration using the registration algorithm, a registration error can be obtained. Therefore, in this step, the above-mentioned first registration matrix can be understood as the registration matrix obtained by the registration with the smallest registration error in the process of selecting coordinates for registration multiple times.

[0171] In this embodiment, the N first reference coordinates of the N markers accurately acquired by the optical measurement device at the first reference time are first determined. If the number K of the K first coordinates acquired at the first time later than the first reference time is less than N, that is, if the markers are occluded, the same number (i.e., M) of coordinates are selected from the K first coordinates and the N first reference coordinates for registration. If the registration is successful, the obtained first registration matrix is ​​used as the registration matrix between the K first coordinates and the N markers. This can save the time spent on the registration process, improve the registration efficiency and accuracy, and thus increase the frequency of real-time updates of surgical navigation.

[0172] To facilitate understanding of the registration between the aforementioned N first reference coordinates and the aforementioned K first coordinates, reuse Figure 4 .like Figure 4 As shown, in this embodiment, time A can be understood as the aforementioned first reference time, time B can be understood as the aforementioned first time, coordinate A can be understood as the coordinates of the aforementioned N markers acquired by the CT device at the first reference time, coordinate B can be understood as the aforementioned N first reference coordinates, and coordinate C can be understood as the aforementioned K first coordinates. It should be understood that when the aforementioned N markers are occluded, K is less than N, meaning that the coordinates included in coordinate C are less than the coordinates included in coordinate B. Directly using the registration algorithm (the aforementioned ICP algorithm) to register coordinates B and C will result in increased registration time or even registration failure.

[0173] In this embodiment, the same number of coordinates are selected from coordinates B and coordinates C respectively for registration, resulting in a registration matrix M. h Registration matrix M h This can be understood as the first registration matrix mentioned above. Using the first registration matrix as the registration matrix between the K first coordinates and the N first reference coordinates can be understood as registering the attitude position corresponding to time B in the coordinate system of the optical measurement device to the attitude position corresponding to time A, such as... Figure 4 As shown, matrix M0 is the registration matrix at time A that aligns the coordinate system of the optical measurement equipment to the coordinate system corresponding to the CT equipment. In other words, matrix M0 is the registration matrix between different coordinate systems. h It is the registration matrix between different times and different attitude positions in the same coordinate system. At time B, through the registration matrix M... h After accurately registering coordinate C to coordinate B, the registration matrix M (i.e., M = M0 × M) can be used. h Registration is performed between the coordinate system corresponding to the CT equipment and the coordinate system corresponding to the optical measurement equipment.

[0174] In some embodiments, Figure 5 In the method shown, the method further includes the following step before step 503:

[0175] 4. Obtain the historical registration matrix, which is the registration matrix between the N first reference coordinates and the N second reference coordinates. The N second reference coordinates are the coordinates of the N markers collected by the optical measuring device at the second reference time. The second reference time is later than the first reference time and earlier than the first time. The time interval between the first reference time and the second reference time is less than or equal to the first time threshold.

[0176] Step 503 includes:

[0177] 5. Based on the historical registration matrix, obtain the first error of registering the K first coordinates to the N first reference coordinates;

[0178] 6. If the first error is greater than the first threshold, the registration process is performed on the M first reference coordinates and the M first coordinates based on the registration algorithm;

[0179] 7. If the first error is less than the first threshold, the historical registration matrix shall be used as the registration matrix between the K first coordinates and the N first reference coordinates.

[0180] In this embodiment, the second reference time is after the first reference time and before the first reference time, and the time interval between the first reference time and the second reference time is less than or equal to the first time threshold. The first time threshold can be set according to the actual situation. For example, the first time threshold can be 1 minute, 2 minutes, etc.

[0181] For example, please refer to Figure 8 , Figure 8 This is a schematic diagram illustrating how to determine a historical registration matrix according to an embodiment of this application. Figure 8 As shown, the optical measuring device accurately acquires N first reference coordinates of N markers at a first reference time, and accurately acquires N second reference coordinates of the N markers at a second reference time. It can be understood that the second reference time is very close to the first reference time, meaning the N markers are not obstructed, and the optical measuring device can accurately acquire their coordinates. For example, if the first reference time is the moment of breath-holding, then the second reference time can be understood as a moment very close to the moment of breath-holding after the breath-holding ends.

[0182] In this embodiment, a registration matrix is ​​obtained by registering the above-mentioned N first reference coordinates and the above-mentioned N second reference coordinates using a registration algorithm (such as the ICP algorithm). Figure 4 The registration matrix Ms in the above can be understood as the historical registration matrix mentioned above. Additionally, as... Figure 8 As shown, the K first coordinates acquired by the optical measuring device at the first moment are registered with the aforementioned N first reference coordinates through a registration matrix M. h Registration is performed. The time difference between the first time point and the first reference time point is greater than the time difference between the second reference time point and the first reference time point.

[0183] In this embodiment of the application, if the registration process in step 504 is successful, the registration device updates the historical registration matrix obtained at the second reference time using the first registration matrix obtained from the registration process, thereby improving the efficiency of real-time registration. It should be understood that the historical registration matrix obtained at subsequent times after the first time is the updated historical registration matrix.

[0184] In this embodiment, after obtaining the aforementioned K first coordinates, the first error of registering the K first coordinates to the N first reference coordinates is obtained based on the historical registration matrix. The aforementioned K first coordinates can be directly obtained, or they can be obtained after artifact point processing.

[0185] In this step, the first error of registering the K first coordinates to the N first reference coordinates is obtained based on the historical registration matrix. This can be understood as first using the historical registration matrix to transform the K first coordinates to obtain K processed first coordinates. Since K is less than N, the first error can be understood as the distance between the K processed first coordinates and the corresponding K first reference coordinates among the N first reference coordinates. If this distance is greater than a first threshold, the registration process is performed on the M first reference coordinates and the M first coordinates based on the registration algorithm. For example, the first threshold can be set according to the actual situation, such as 2 mm, 3 mm, etc.

[0186] In this embodiment, the K first coordinates are registered to the coordinate system of the N first reference coordinates using a historical registration matrix. Only when the registration error is greater than a first threshold, that is, when the first error obtained by registering coordinates using the historical registration matrix is ​​greater than the first threshold, is the registration algorithm used to perform the registration process on the M first reference coordinates and the M first coordinates. When the first error is less than the first threshold, the historical registration matrix is ​​used directly for registration, which can save registration time and improve registration efficiency.

[0187] It should be understood that in this embodiment, the first error equal to the first threshold can be categorized into any of the above branches. Exemplarily, optionally, if the first error is greater than or equal to the first threshold, the registration process is performed on the M first reference coordinates and the M first coordinates based on the registration algorithm; if the first error is less than the first threshold, the historical registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates.

[0188] Optionally, if the first error is greater than the first threshold, the registration process is performed on the M first reference coordinates and the M first coordinates based on the registration algorithm; if the first error is less than or equal to the first threshold, the historical registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates.

[0189] In some embodiments, the method shown in the preceding embodiment further includes:

[0190] If the registration process fails, the historical registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates.

[0191] In this embodiment, the registration device obtains the first error of registering the K first coordinates to the N first reference coordinates based on the historical registration matrix; if the first error is greater than the first threshold, the same number of coordinates are selected from the K first coordinates and the N first reference coordinates for registration; if the registration fails, the registration device uses the historical registration matrix as the registration matrix between the K first coordinates and the N first reference coordinates to improve the robustness of surgical guidance.

[0192] In some embodiments, Figure 5 The method shown also includes:

[0193] If the registration process fails, the historical registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates.

[0194] In this embodiment, the method for obtaining the historical registration matrix can be referred to the preceding text. Figure 8 The corresponding embodiments will not be described in detail here. In this embodiment, the registration device selects the same number of coordinates from the K first coordinates and the N first reference coordinates for registration; in the event of registration failure, the registration device obtains the historical registration matrix and uses the historical registration matrix as the registration matrix between the K first coordinates and the N first reference coordinates to improve the robustness of surgical guidance.

[0195] In some embodiments, step 503 above includes:

[0196] 5031. Based on the historical registration matrix, perform a first registration process on the M first coordinates to obtain M processed first coordinates. The first distance is less than the second distance. The first distance is the average of the first Euclidean distances between each of the M processed first coordinates and the corresponding coordinates in the N first reference coordinates. The second distance is the average of the second Euclidean distances between the M first coordinates and the corresponding coordinates in the N first reference coordinates.

[0197] 5032. Based on the registration algorithm, perform a second registration process on the M first reference coordinates and the M processed first coordinates.

[0198] It is understandable that the number of coordinates M in the M processed first coordinates is less than the number of coordinates N in the N first reference coordinates. Therefore, M coordinates in the N first reference coordinates correspond to the coordinates in the M processed first coordinates. The aforementioned first Euclidean distance should be understood as the set of Euclidean distances between the M sets of corresponding coordinates. For example, the correspondence between coordinates can be obtained using the ICP algorithm. M candidate coordinates are selected from the N first reference coordinates and registered with the M processed first coordinates. The M candidate coordinates from the registration process with the smallest registration error are taken as the coordinate set corresponding to the M processed first coordinates. Coordinates with the same index number in the two coordinate sets can be considered corresponding coordinates. It should be understood that the aforementioned second Euclidean distance is similar to the aforementioned first Euclidean distance, and will not be elaborated upon here.

[0199] In this embodiment, the method for obtaining the historical registration matrix can be referred to the preceding text. Figure 8 The relevant descriptions will not be repeated here. The first registration process described above can be understood as using the acquired historical registration matrix to transform the above M first coordinates to obtain M processed first coordinates. It should be understood that, in one possible implementation, after obtaining the above K first coordinates, the registration device transforms the above K first coordinates based on the acquired historical registration matrix to obtain K processed first coordinates, and then selects M processed first coordinates from them. In another possible implementation, the configuration device selects M first coordinates from the above K first coordinates, and then transforms the above M first coordinates based on the acquired historical registration matrix to obtain M processed first coordinates.

[0200] In this embodiment, the M first coordinates are first registered using the historical registration matrix. The distance between the M processed first coordinates and the N first reference coordinates is less than the distance between the K first coordinates and the N first reference coordinates. This makes the M processed first coordinates closer to the M first reference coordinates, thereby improving the efficiency of the registration process in the subsequent second registration process.

[0201] It is understood that steps 5031 and 5032 described above can be coupled with any of the embodiments mentioned above that include step 503. In each embodiment, the method for obtaining the historical registration matrix can be found in the preceding text. Figure 8 Related descriptions.

[0202] To facilitate a comprehensive understanding of the methods provided in the embodiments of this application, please refer to the following sections. Figure 9 , Figure 9 This is a flowchart illustrating another registration method provided in an embodiment of this application. For example... Figure 9 As shown, the method includes:

[0203] 901: Obtain the coordinate set P(P1, P2, ..., P...) N ), where the coordinate set P is the coordinates of N markers collected by the optical measuring equipment at a standard time.

[0204] In this step, the aforementioned standard time can be understood as the aforementioned first reference time, as detailed in the relevant description of step 501 above. In this step, the coordinate set P contains N coordinates, which correspond to N markers. The coordinate set P can be understood as the aforementioned N first reference coordinates.

[0205] 902: Calculate the normal vector and triangle perimeter for every three coordinates in coordinate set P to obtain a classification template. The normal vector and triangle perimeter in the classification template correspond to each other.

[0206] In this step, every three coordinates out of the N coordinates can form a triangle. Using this triangle, the normal vector corresponding to those three coordinates and the perimeter of the triangle can be calculated. Therefore, N coordinates can yield a set of normal vectors and triangle perimeters. Since three coordinates can simultaneously determine a normal vector and a triangle perimeter, the correspondence between the normal vector and the triangle perimeter can be understood as the normal vector and the triangle perimeter originating from the same coordinates.

[0207] 903: Obtain the coordinate set L(L1, L2, ..., L...) N ), where the coordinate set L is the coordinates of the N markers acquired by the CT equipment at a standard time.

[0208] In this step, the coordinate set L contains N coordinates, each corresponding to one of the N markers. The coordinate set L is obtained by scanning the N markers at a standard time to obtain a CT image containing those N markers, and then segmenting the markers from the CT image.

[0209] 904: Calculate the registration matrix M0 of coordinate sets P and L at the standard time.

[0210] In this step, the number of coordinates in coordinate set P is the same as the number of coordinates in coordinate set L. The registration matrix M0 of coordinate set P and coordinate set L at the standard time can be obtained by the ICP algorithm.

[0211] 905: Get coordinate set A(A1, A2, ..., A...) H The coordinate set P is the coordinates of the N markers collected by the optical measuring device at a time after the standard time.

[0212] In this step, the time following the standard time can be understood as the first time mentioned above. It should be understood that in this step, N markers may be obscured, and artifacts may occur during the acquisition of coordinates by the optical measuring equipment. Therefore, H in this step may be greater than N, equal to N, or less than N.

[0213] In this step, the coordinate set A can be understood as the H original first coordinates mentioned above. For details, please refer to step 5021 above.

[0214] 906: Select coordinates A1, A2, and A3 from coordinate set A.

[0215] 907: Calculate the normal vector A and the perimeter A of the triangle based on coordinates A1, A2, and A3.

[0216] 908: Determine whether the angle between normal vectors A and B is less than the threshold A, and whether the absolute value of the difference between the perimeter of triangle A and the perimeter of triangle B corresponding to normal vector A is less than the threshold B. Normal vector X is any normal vector in the classification template.

[0217] In this step, the threshold A mentioned above can be understood as the second threshold mentioned above, and the threshold B can be understood as the third threshold mentioned above.

[0218] If the judgment result in step 908 is "yes", then step 909 is executed: coordinates A1, coordinates A2 and coordinates A3 are used as coordinates in the first processed coordinate set A.

[0219] In this embodiment, the first processed coordinate set A can be understood as the coordinate set after removing the coordinates of the artifact points from the coordinate set A. That is, the first processed coordinate set A can be understood as the coordinates of the real marker.

[0220] If the result of the judgment in step 908 is "no", then step 906 is executed again.

[0221] It should be understood that in this step, when step 906 is re-executed, three coordinates other than coordinates A1, A2, and A3 will be selected from coordinate set A.

[0222] After step 909, execute 9010: based on coordinates A1, coordinates A2, and coordinates A i Calculate the normal vector B and the perimeter B of the triangle, and the coordinates A. i It is any coordinate in coordinate set A other than coordinates A1, A2, and A3.

[0223] 9011: Determine whether the angle between normal vector B and normal vector X is less than threshold A, and whether the absolute value of the difference between the perimeter of triangle B and the perimeter Y of the triangle corresponding to normal vector X is less than threshold B. Normal vector X is any normal vector in the classification template.

[0224] If the judgment result in step 9011 is "yes", then proceed to step 9012: set coordinate A. i As coordinates in the first processed coordinate set A.

[0225] In this step, the coordinate A i This can be understood as the coordinates of a real, existing marker.

[0226] If the judgment result of step 9011 is "no", then execute step 9013: determine whether all coordinates in coordinate set A except coordinates A1, coordinates A2 and coordinates A3 have been traversed.

[0227] If the judgment result in step 9013 is "yes", then execute step 9014: obtain the first processed coordinate set A(A1, A2, ..., A...). K ), where K is less than or equal to N.

[0228] It should be understood that after all coordinates in coordinate set A except for A1, A2, and A3 have been traversed, the artifact point removal process for coordinate set A can be considered complete, and the K coordinates in the resulting first-processed coordinate set A are the truly existing coordinates. It should also be understood that after artifact point removal, markers may be occluded; therefore, the number of coordinates K in the first-processed coordinate set A may be less than or equal to N.

[0229] In this step, the first processed coordinate set A can be understood as the aforementioned K first coordinates.

[0230] If the result of the judgment in step 9013 is "no", then step 9010 is executed again.

[0231] In this step, the coordinates in coordinate set A other than A1, A2 and A3 have not been fully traversed, so we continue to process the other coordinates in coordinate set A.

[0232] In this embodiment, steps 905-9013 can also refer to the relevant descriptions of steps 5021-5022 above.

[0233] After step 9014, proceed to step 9015: Obtain the historical registration matrix M. h .

[0234] In this step, the method for obtaining the historical registration matrix can be found in the previous text. Figure 8 The relevant descriptions will not be repeated here.

[0235] 9016: Determine if K is less than N.

[0236] In this step, K is less than N, which can be understood as some of the N markers being obscured.

[0237] If the judgment result in step 9016 is "yes", then proceed to step 9017: using the historical registration matrix M h Perform operations on the processed coordinate set A to obtain the second processed coordinate set A.

[0238] If the judgment result of step 9016 is "no", then proceed to step 9023: transfer the historical registration matrix M. h As the registration matrix between coordinate set A and coordinate set P.

[0239] In this step, assuming that N markers are not occluded, the historical registration matrix M is... h This serves as the registration matrix between coordinate set A and coordinate set P. Alternatively, in some embodiments, since the number of coordinates in coordinate set A and coordinate set P is equal, the registration matrix between coordinate set A and coordinate set P can be directly calculated using a registration algorithm.

[0240] After step 9017, step 9018 is executed: calculate the error between the second-processed coordinate set A and the coordinate set P.

[0241] In this embodiment, the second processed coordinate set A can be understood as using the historical registration matrix M h The coordinate set obtained by performing operations on the first processed coordinate set A.

[0242] 9019: Determine whether the error is less than the threshold C.

[0243] In this step, the threshold C can be understood as the first threshold mentioned above. The error in this step can be understood as the first error mentioned above.

[0244] If the judgment result of step 9019 is "yes", then step 9020 is executed: S coordinates are taken from the second processed coordinate set A and coordinate set P respectively for registration, where S is a number greater than or equal to 3.

[0245] For a description of this step, please refer to the relevant embodiment description of step 503 above.

[0246] If the judgment result of step 9019 is "no", then proceed to step 9023: transfer the historical registration matrix M. h As the registration matrix between coordinate set A and coordinate set P.

[0247] After step 9020, proceed to step 9021: determine whether registration was successful.

[0248] This step can be understood as determining whether the registration was successful in step 9020, which involves taking S coordinates from the second-processed coordinate set A and coordinate set P respectively.

[0249] If the judgment result in step 9021 is "yes", then step 9022 is executed: the registration matrix M1 between coordinate set A and coordinate set P is obtained.

[0250] It should be understood that the registration matrix M1 is the registration matrix obtained by taking S coordinates from the coordinate set A and coordinate set P after the second processing and registering them.

[0251] If the judgment result of step 9021 is "no", then proceed to step 9023: transfer the historical registration matrix M. h As the registration matrix between coordinate set A and coordinate set P.

[0252] In this embodiment, steps 9015-9023 can refer to the previous description of step 503 and steps 4-7 related to step 503.

[0253] After step 9022 or step 9023, perform step 9024: based on the registration matrix M1, or the historical registration matrix M h The registration matrix M is obtained at time A, which registers the coordinate system corresponding to the optical measurement device to the coordinate system corresponding to the CT device.

[0254] In this step, if the previous step was step 9022, i.e., the registration matrix M1 is the registration matrix between coordinate set A and coordinate set P, then the registration matrix M is equal to M1 × M. h If the previous step was step 9023, then the registration matrix M is equal to the historical registration matrix M. h .

[0255] 9025: Inverting the registration matrix M yields the registration matrix that registers the coordinate system corresponding to the CT device to the coordinate system corresponding to the optical measurement device at time A.

[0256] Understandably, in actual surgery, the surgical needle is positioned using optical measurement equipment and aligned with the model under the CT scan. By inverting the registration matrix M, the internal tissue model of the surgical object in the coordinate system corresponding to the CT scan is registered to the coordinate system corresponding to the optical measurement equipment, allowing for a more consistent display of the surgical needle's movements.

[0257] The methods provided in the embodiments of this application have been described in detail above. The apparatus provided in the embodiments of this application is described below.

[0258] Please see Figure 10 , Figure 10 This is a schematic diagram of a registration device provided in an embodiment of this application. The registration device 100 is used to execute the coordinate determination method described above. It should be understood that any device capable of implementing the coordinate determination method provided in this application falls within the protection scope of this application. Exemplarily, the registration device 100 can be a mobile phone, desktop computer, or laptop, etc., and this embodiment of the application does not limit it. Figure 10 As shown, the registration device 100 includes a determining unit 1001, an acquiring unit 1002, and a registration unit 1003. Optionally, the registration device 100 may further include a response unit 1004. The descriptions of each unit are as follows:

[0259] The determining unit 1001 is used to determine N first reference coordinates, wherein the N first reference coordinates are the coordinates of N markers collected by the optical measuring device at a reference time, and N is an integer greater than or equal to 3;

[0260] The acquisition unit 1002 is used to acquire K first coordinates, which are determined based on the coordinates of the N markers collected by the optical measuring device at a first moment. The first moment is later than the reference moment, and K is an integer greater than or equal to 3.

[0261] The registration unit 1003 is used to perform registration processing on M first reference coordinates and M first coordinates based on a registration algorithm. The M first reference coordinates are selected from the N first reference coordinates, and the M first coordinates are selected from the K first coordinates. M is an integer greater than or equal to 3.

[0262] The determining unit 1001 is further configured to, when the registration process results in successful registration, use the first registration matrix as the registration matrix between the K first coordinates and the N first reference coordinates, wherein the first registration matrix is ​​obtained through the registration process and is the registration matrix between the M first reference coordinates and the M first coordinates.

[0263] In one possible implementation, the registration device further includes a response unit 1004, which responds to a first trigger operation and controls the determination unit 1001 to determine the N first reference coordinates. The first trigger operation is used to indicate that the implementation object is in a first state, and the surface of the implementation object is covered with the N markers.

[0264] The response unit 1004 is further configured to respond to the second trigger operation and control the acquisition unit 1002 to acquire the K first coordinates. The second trigger operation is configured to indicate that the implementation object is in a second state, and the stability of the first state is greater than the stability of the second state.

[0265] In one possible implementation, the acquisition unit 1002 is further configured to acquire a historical registration matrix, wherein the historical registration matrix is ​​a registration matrix between the N first reference coordinates and the N second reference coordinates, wherein the N second reference coordinates are the coordinates of the N markers acquired by the optical measuring device at a second reference time, wherein the second reference time is later than the first reference time, and the second reference time is earlier than the first time, and the time interval between the first reference time and the second reference time is less than or equal to a first time threshold.

[0266] The aforementioned acquisition unit 1002 is specifically used to acquire the first error of registering the aforementioned K first coordinates to the aforementioned N first reference coordinates based on the aforementioned historical registration matrix;

[0267] The registration unit 1003 is specifically used to perform the registration process on the M first reference coordinates and the M first coordinates based on the registration algorithm when the first error is greater than the first threshold.

[0268] The registration unit 1003 is specifically used to use the historical registration matrix as the registration matrix between the K first coordinates and the N first reference coordinates when the first error is less than the first threshold.

[0269] In one possible implementation, the determining unit 1001 is further configured to use the historical registration matrix as the registration matrix between the K first coordinates and the N first reference coordinates when the registration process results in registration failure.

[0270] In one possible implementation, the registration unit 1003 is specifically used to perform a first registration process on the M first coordinates based on the historical registration matrix to obtain M processed first coordinates, where the first distance is less than the second distance. The first distance is the average of the first Euclidean distances between each of the M processed first coordinates and the corresponding coordinates in the N first reference coordinates, and the second distance is the average of the second Euclidean distances between the coordinates in the M first coordinates and the corresponding coordinates in the N first reference coordinates.

[0271] The registration unit 1003 is specifically used to perform a second registration process on the M first reference coordinates and the M processed first coordinates based on the registration algorithm.

[0272] In one possible implementation, the acquisition unit 1002 is specifically used to acquire H original first coordinates, wherein the H original first coordinates are the coordinates of the N markers collected by the optical measuring device at the first moment, and H is an integer greater than K.

[0273] The aforementioned determining unit 1001 is specifically used to determine the aforementioned K first coordinates from the aforementioned H original first coordinates, wherein the included angle between the first plane and the second plane is less than or equal to a second threshold, the aforementioned first plane is obtained based on three coordinates from the aforementioned K first coordinates, and the aforementioned second plane is obtained based on three coordinates from the aforementioned N first reference coordinates.

[0274] In one possible implementation, the absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to a third threshold, wherein the first figure is a triangle formed by three coordinates corresponding to the first plane, and the second figure is a triangle formed by three coordinates corresponding to the second plane.

[0275] Please see Figure 11 , Figure 11 This is a schematic diagram of another registration device provided in an embodiment of this application. The registration device 110 can be used to implement the above-described registration method. For example, the registration device 100 can be a mobile phone, desktop computer, laptop, or other similar device.

[0276] like Figure 11 As shown. The registration device 110 includes at least one processor 1102 and a transceiver 1101 for communicating with other devices / appliances via a transmission medium. The processor 1102 can use the transceiver 1101 to send and receive data and / or signaling.

[0277] Optionally, the registration device 110 may further include at least one memory 1103 for storing program instructions and / or data. The memory 1103 is coupled to the processor 1102. The coupling in this embodiment is an indirect coupling or communication connection between devices, units, or modules, and can be electrical, mechanical, or other forms, used for information exchange between devices, units, or modules. The processor 1102 may operate in conjunction with the memory 1103. The processor 1102 may execute program instructions stored in the memory 1103. At least one of the at least one memory may be included in the processor.

[0278] This application embodiment does not limit the specific connection medium between the transceiver 1101, processor 1102, and memory 1103. This application embodiment... Figure 11 The memory 1103, processor 1102, and transceiver 1101 are connected via a bus 1104. Figure 11The connections between other components are shown in thick lines only and are not intended to be limiting. This bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0279] In the embodiments of this application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0280] It is understood that when the registration device 110 is the aforementioned registration device 100, the actions performed by the acquisition unit 1002 can be performed by the transceiver 1101, or by the processor 1102; the actions performed by the determination unit 1001, the registration unit 1003, and the response unit 1004 can be performed by the processor 1102.

[0281] This application also provides a computer-readable storage medium storing computer code that, when executed on a computer, causes the computer to perform the methods described in the above embodiments.

[0282] This application also provides a computer program product, which includes computer code or a computer program that, when run on a computer, causes the methods described in the above embodiments to be executed.

[0283] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of protection of the above claims.

Claims

1. A registration method, characterized in that, The method includes: N first reference coordinates are determined, wherein the N first reference coordinates are the coordinates of N markers collected by the optical measuring device at the first reference time, and N is an integer greater than or equal to 3; Obtaining K first coordinates, wherein the K first coordinates are determined based on the coordinates of the N markers collected by the optical measuring device at a first moment, the first moment being later than the first reference moment, and K being an integer greater than or equal to 3; obtaining the K first coordinates includes: obtaining H original first coordinates, wherein the H original first coordinates are the coordinates of the N markers collected by the optical measuring device at the first moment, and H being an integer greater than K; determining the K first coordinates from the H original first coordinates, wherein the angle between the first plane and the second plane is less than or equal to a second threshold, the first plane being obtained based on three coordinates from the K first coordinates, and the second plane being obtained based on three coordinates from the N first reference coordinates; When K is less than N, a registration process is performed on M first reference coordinates and M first coordinates based on a registration algorithm. The M first reference coordinates are selected from the N first reference coordinates, and the M first coordinates are selected from the K first coordinates. M is an integer greater than or equal to 3. If the registration process results in successful registration, the first registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates. The first registration matrix is ​​obtained through the registration process and is the registration matrix between the M first reference coordinates and the M first coordinates.

2. The method according to claim 1, characterized in that, The determination of N first reference coordinates includes: In response to a first triggering operation, the N first reference coordinates are determined. The first triggering operation is used to indicate that the implementation object is in a first state, and the N markers are pasted on the body surface of the implementation object. The process of obtaining the K first coordinates includes: In response to a second triggering operation, the K first coordinates are obtained. The second triggering operation is used to indicate that the implementation object is in a second state, and the stability of the first state is greater than the stability of the second state.

3. The method according to claim 1 or 2, characterized in that, Before performing registration processing on the M first reference coordinates and the M first coordinates based on the registration algorithm, the method further includes: Obtain a historical registration matrix, which is a registration matrix between the N first reference coordinates and the N second reference coordinates. The N second reference coordinates are the coordinates of the N markers collected by the optical measuring device at a second reference time. The second reference time is later than the first reference time and earlier than the first time. The time interval between the first reference time and the second reference time is less than or equal to a first time threshold. The registration process based on the registration algorithm for M first reference coordinates and M first coordinates includes: Based on the historical registration matrix, obtain the first error for registering the K first coordinates to the N first reference coordinates; If the first error is greater than the first threshold, the registration process is performed on the M first reference coordinates and the M first coordinates based on the registration algorithm; If the first error is less than the first threshold, the historical registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates.

4. The method according to claim 1 or 2, characterized in that, The method further includes: If the registration process results in a registration failure, the historical registration matrix is ​​used as the registration matrix between the K first coordinates and the N first reference coordinates.

5. The method according to claim 4, characterized in that, The registration process based on the registration algorithm for M first reference coordinates and M first coordinates includes: Based on the historical registration matrix, the M first coordinates are subjected to a first registration process to obtain M processed first coordinates. The first distance is less than the second distance. The first distance is the average of the first Euclidean distances between each of the M processed first coordinates and the corresponding coordinates in the N first reference coordinates. The second distance is the average of the second Euclidean distances between the M first coordinates and the corresponding coordinates in the N first reference coordinates. Based on the registration algorithm, a second registration process is performed on the M first reference coordinates and the M processed first coordinates.

6. The method according to claim 1, characterized in that, The absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure is less than or equal to a third threshold. The first figure is a triangle formed by the three coordinates corresponding to the first plane, and the second figure is a triangle formed by the three coordinates corresponding to the second plane.

7. A registration device, characterized in that, The device includes: A determining unit is used to determine N first reference coordinates, wherein the N first reference coordinates are the coordinates of N markers collected by the optical measuring device at a reference time, and N is an integer greater than or equal to 3; An acquisition unit is configured to acquire K first coordinates, wherein the K first coordinates are determined based on the coordinates of the N markers acquired by the optical measuring device at a first moment, the first moment being later than the reference moment, and K being an integer greater than or equal to 3; acquiring the K first coordinates includes: acquiring H original first coordinates, wherein the H original first coordinates are the coordinates of the N markers acquired by the optical measuring device at the first moment, and H being an integer greater than K; determining the K first coordinates from the H original first coordinates, wherein the angle between a first plane and a second plane is less than or equal to a second threshold, the first plane being obtained based on three coordinates from the K first coordinates, and the second plane being obtained based on three coordinates from the N first reference coordinates; A registration unit is used to perform registration processing on M first reference coordinates and M first coordinates based on a registration algorithm. The M first reference coordinates are selected from the N first reference coordinates, and the M first coordinates are selected from the K first coordinates. M is an integer greater than or equal to 3. The determining unit is further configured to, if the registration process results in successful registration, use the first registration matrix as the registration matrix between the K first coordinates and the N first reference coordinates, wherein the first registration matrix is ​​obtained through the registration process and is the registration matrix between the M first reference coordinates and the M first coordinates.

8. An electronic device, characterized in that, include: A processor and a memory, wherein the memory stores a computer program, and the processor invokes the computer program stored in the memory to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on one or more processors, causes the method as described in any one of claims 1-6 to be performed.

Images