Graph registration-based method, surgical robot and related product

Abstract

The invention discloses a method based on graph registration, a surgical robot and a related product. The method comprises the steps that n first coordinates and n second coordinates are obtained, the n first coordinates are coordinates of n reference objects in a first coordinate system, and the n second coordinates are coordinates of the n reference objects in a second coordinate system; a first coordinate set is determined from the n first coordinates. A second coordinate set is determined from the n second coordinates. And obtaining a first graph based on the coordinates in the first coordinate set. And obtaining a second graph based on the coordinates in the second coordinate set, wherein the shape of the first graph is the same as that of the second graph. And based on the first graph and the second graph, obtaining a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set. The invention further discloses a corresponding product. Based on the method provided by the invention, the efficiency of determining the matching relationship between the coordinates under different coordinate systems can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a method based on graphic registration, a surgical robot and related products. BACKGROUND

[0002] In the medical field, it is often necessary to convert coordinates in different coordinate systems. Therefore, it is necessary to determine the registration relationship between different coordinate systems, wherein the registration relationship is used to convert the coordinates in one coordinate system into the coordinates in another coordinate system.

[0003] The current technology usually determines the matching relationship between the coordinates in different coordinate systems first, and then calculates the registration relationship between the different coordinate systems based on the matching relationship.

[0004] However, the efficiency of determining the matching relationship between the coordinates in different coordinate systems by the current technology is low. SUMMARY

[0005] The present application provides a method based on graphic registration, a surgical robot and related products, wherein the related products include a device based on graphic registration and an electronic device, so as to improve the efficiency of determining the matching relationship between the coordinates in different coordinate systems.

[0006] In a first aspect, a method based on graphic registration is provided, and the method comprises: obtaining n first coordinates and n second coordinates, the n first coordinates being the coordinates of n reference objects in a first coordinate system, the n second coordinates being the coordinates of the n reference objects in a second coordinate system, and the n being an integer greater than 3; determining a first coordinate set from the n first coordinates, the first coordinate set comprising m first coordinates in the n first coordinates, and the m being an integer greater than or equal to 3 and less than or equal to the n; determining a second coordinate set from the n second coordinates, the second coordinate set comprising m second coordinates in the n second coordinates; obtaining a first graph based on the coordinates in the first coordinate set; obtaining a second graph based on the coordinates in the second coordinate set, the shape of the first graph being the same as the shape of the second graph; obtaining a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first graph and the second graph.

[0007] In combination with any embodiment of the present application, the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first graph and the second graph comprises: determining an absolute value of a difference between the perimeter of the first graph and the perimeter of the second graph, to obtain a first absolute value; In a case where the first condition is met, determining the matching relationship comprises m groups of matching coordinate pairs; The first condition comprises that the first absolute value is less than or equal to a first threshold value. The matching coordinate pair comprises one coordinate in the first coordinate set and one coordinate in the second coordinate set, and the two coordinates in the matching coordinate pair correspond to the same one of the n references. The m groups of matching coordinate pairs are obtained based on positions of the coordinates in the first coordinate set in the first graph and positions of the coordinates in the second coordinate set in the second graph. In a case where the first condition is not met, determining that the matching relationship is that there is no matching coordinate pair in the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0008] In combination with any embodiment of the present application, the m is 3, and shapes of the first graph and the second graph are both triangles. The first condition further comprises at least two of the following: a second absolute value is less than or equal to a second threshold value, a third absolute value is less than or equal to a third threshold value, and a fourth absolute value is less than or equal to a fourth threshold value. The second absolute value is an absolute value of a difference between a largest internal angle in the first graph and a largest internal angle in the second graph. The third absolute value is an absolute value of a difference between a second largest internal angle in the first graph and a second largest internal angle in the second graph. The fourth absolute value is an absolute value of a difference between a smallest internal angle in the first graph and a smallest internal angle in the second graph.

[0009] In this embodiment, if two triangles are similar, the perimeters of the two triangles should differ by less than or equal to the first threshold value, the largest internal angles of the two triangles should differ by less than or equal to the second threshold value, the second largest internal angles of the two triangles should differ by less than or equal to the third threshold value, and the smallest internal angles of the two triangles should differ by less than or equal to the fourth threshold value. Therefore, the accuracy of determining whether the two first graphs and the second graph are similar based on the first condition can be improved, and the accuracy of the matching relationship can be improved.

[0010] In combination with any embodiment of the present application, the method further comprises: In a case where the matching relationship comprises the m groups of matching coordinate pairs, a first registration relationship between the first coordinate system and the second coordinate system is obtained based on the m groups of matching coordinate pairs. The first registration relationship is used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0011] In combination with any of the embodiments of the present application, the m is less than the n, and the method further comprises: translating a third coordinate based on the first registration relationship to obtain a fourth coordinate, the third coordinate being one of the n first coordinates other than the coordinates in the first coordinate set; determining n-m second coordinates of the n second coordinates other than the coordinates in the second coordinate set; in a case where a fifth coordinate exists in the n-m second coordinates, the fifth coordinate being less than or equal to a fifth threshold value from the third coordinate, obtaining a set of matching coordinate pairs, the set of matching coordinate pairs including the third coordinate and the fifth coordinate; based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, obtaining a second registration relationship of the first coordinate system and the second coordinate system, the second registration relationship being used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0012] In combination with any of the embodiments of the present application, the first coordinate system is an image coordinate system of a first computed tomography (CT) image, the first CT image being an image collected by a CT collection device at a first time, the second coordinate system is a coordinate system established based on an optical tracking device, the n reference objects are n optical markers attached to a body surface of a first object, and the n second coordinates are determined by the optical tracking device at the first time. The method further comprises: based on the second registration relationship, converting the n first coordinates into n first converted coordinates in the second coordinate system; based on a difference between the n first converted coordinates and the n second coordinates, obtaining a first error of the second registration relationship, the difference being positively correlated with the first error; obtaining a second error of a reference registration relationship, the reference registration relationship being used to convert a coordinate in an image coordinate system of a second CT image into a coordinate in the second coordinate system, the second CT image being collected by the CT collection device at a second time, the reference registration relationship being obtained based on positions of the n optical markers in the image coordinate system of the second CT image and n sixth coordinates of the n optical markers in the second coordinate system, the n sixth coordinates being determined by the optical tracking device at the second time; based on a difference between the first error and the second error, determining a difference between a breathing state of the first object at the first time and a breathing state of the first object at the second time.

[0013] In combination with any of the embodiments of the present application, before the second registration relationship of the first coordinate system and the second coordinate system is obtained based on the m sets of matched coordinate pairs and the set of matched coordinate pairs, the method further comprises: obtaining a seventh coordinate of a first optical marker in the n optical markers, the seventh coordinate being determined by the optical tracking device at a third time, a time difference between the first time and the third time being less than a sixth threshold value, the sixth threshold value being based on a duration of a breathing cycle of the first object; determining an eighth coordinate from the n second coordinates, the eighth coordinate being a coordinate of the first optical marker in the second coordinate system; determining an absolute value of a difference between the seventh coordinate and the eighth coordinate to obtain a fifth absolute value; obtaining, based on the fifth absolute value, a constraint condition for the registration relationship of the first coordinate system and the second coordinate system, the constraint condition including that an absolute value of a difference between a second conversion coordinate and the eighth coordinate is less than or equal to a seventh threshold value, the second conversion coordinate being obtained by converting the seventh coordinate into a coordinate in the second coordinate system based on the registration relationship of the first coordinate system and the second coordinate system, the seventh threshold value being positively correlated with the fifth absolute value; the second registration relationship of the first coordinate system and the second coordinate system is obtained based on the m sets of matched coordinate pairs and the set of matched coordinate pairs, including: the second registration relationship is obtained based on the constraint condition, the m sets of matched coordinate pairs and the set of matched coordinate pairs.

[0014] In a second aspect, a device based on graphic registration is provided. The device based on graphic registration comprises: an obtaining unit configured to obtain n first coordinates and n second coordinates, the n first coordinates being coordinates of n reference objects in a first coordinate system, the n second coordinates being coordinates of the n reference objects in a second coordinate system, n being an integer greater than 3; a processing unit configured to determine a first coordinate set from the n first coordinates, the first coordinate set including m first coordinates in the n first coordinates, m being an integer greater than or equal to 3 and less than or equal to n; the processing unit is further configured to determine a second coordinate set from the n second coordinates, the second coordinate set including m second coordinates in the n second coordinates; the processing unit is further configured to obtain a first graphic based on coordinates in the first coordinate set; The processing unit is further configured to obtain a second graph based on the coordinates in the second coordinate set, the first graph and the second graph having the same shape. The processing unit is further configured to obtain a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first graph and the second graph.

[0015] In combination with any of the embodiments of the present application, the processing unit is further configured to: obtain a first absolute value by determining a difference between the perimeter of the first graph and the perimeter of the second graph; In a case where a first condition is met, the matching relationship includes m groups of matching coordinate pairs; The first condition includes that the first absolute value is less than or equal to a first threshold value, the matching coordinate pair includes one coordinate in the first coordinate set and one coordinate in the second coordinate set, and the two coordinates in the matching coordinate pair correspond to the same one of the n reference objects, and the m groups of matching coordinate pairs are obtained based on the positions of the coordinates in the first coordinate set in the first graph and the positions of the coordinates in the second coordinate set in the second graph. In a case where the first condition is not met, the matching relationship is that there is no matching coordinate pair between the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0016] In combination with any of the embodiments of the present application, the m is 3, and the shapes of the first graph and the second graph are both triangles. The first condition further includes at least two of the following: a second absolute value is less than or equal to a second threshold value, a third absolute value is less than or equal to a third threshold value, and a fourth absolute value is less than or equal to a fourth threshold value. The second absolute value is an absolute value of a difference between a largest interior angle in the first graph and a largest interior angle in the second graph, the third absolute value is an absolute value of a difference between a second largest interior angle in the first graph and a second largest interior angle in the second graph, and the fourth absolute value is an absolute value of a difference between a smallest interior angle in the first graph and a smallest interior angle in the second graph.

[0017] In combination with any of the embodiments of the present application, the processing unit is further configured to: In a case where the matching relationship includes the m groups of matching coordinate pairs, the processing unit is further configured to obtain a first registration relationship between the first coordinate system and the second coordinate system based on the m groups of matching coordinate pairs, the first registration relationship being used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0018] In combination with any of the embodiments of the present application, the m is less than the n, and the processing unit is further configured to: convert a third coordinate based on the first registration relationship to obtain a fourth coordinate, the third coordinate being one of the n first coordinates other than the coordinates in the first coordinate set; determine n-m second coordinates of the n second coordinates other than the coordinates in the second coordinate set; in a case where a fifth coordinate exists in the n-m second coordinates, the fifth coordinate having a distance to the third coordinate less than or equal to a fifth threshold, obtain a group of matching coordinate pairs, the group of matching coordinate pairs including the third coordinate and the fifth coordinate; obtain a second registration relationship between the first coordinate system and the second coordinate system based on the m groups of matching coordinate pairs and the group of matching coordinate pairs, the second registration relationship being used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0019] In combination with any of the embodiments of the present application, the first coordinate system is an image coordinate system of a first CT image, the first CT image being an image collected by a CT collection device at a first time, the second coordinate system is a coordinate system established based on an optical tracking device, the n reference objects are n optical markers attached to a body surface of a first subject, and the n second coordinates are determined by the optical tracking device at the first time. The processing unit is further configured to convert the n first coordinates into n first converted coordinates in the second coordinate system based on the second registration relationship. The processing unit is further configured to obtain a first error of the second registration relationship based on a difference between the n first converted coordinates and the n second coordinates, the difference being positively correlated with the first error. The acquisition unit is further configured to acquire a second error of a reference registration relationship, the reference registration relationship being used to convert a coordinate in an image coordinate system of a second CT image into a coordinate in the second coordinate system, the second CT image being collected by the CT collection device at a second time, the reference registration relationship being obtained based on positions of the n optical markers in the image coordinate system of the second CT image and n sixth coordinates of the n optical markers in the second coordinate system, the n sixth coordinates being determined by the optical tracking device at the second time. The processing unit is further configured to determine a difference between a breathing state of the first subject at the first time and a breathing state of the first subject at the second time based on a difference between the first error and the second error.

[0020] With any of the embodiments of the present application, the acquisition unit is further configured to acquire a seventh coordinate of a first optical marker in the n optical markers, the seventh coordinate being determined by the optical tracking device at a third time, a time difference between the first time and the third time being less than a sixth threshold value, the sixth threshold value being based on a duration of a breathing cycle of the first object. The processing unit is further configured to determine an eighth coordinate from the n second coordinates, the eighth coordinate being a coordinate of the first optical marker in the second coordinate system. The processing unit is further configured to determine an absolute value of a difference between the seventh coordinate and the eighth coordinate, obtaining a fifth absolute value. The processing unit is further configured to obtain, based on the fifth absolute value, the constraint condition as a constraint condition of a registration relationship between the first coordinate system and the second coordinate system, the constraint condition including that an absolute value of a difference between a second conversion coordinate and the eighth coordinate is less than or equal to a seventh threshold value, the second conversion coordinate being obtained by converting the seventh coordinate into a coordinate in the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system, the seventh threshold value being positively correlated with the fifth absolute value. The processing unit is further configured to obtain the second registration relationship based on the constraint condition, the m sets of matching coordinate pairs, and the set of matching coordinate pairs.

[0021] In a third aspect, a surgical robot is provided, including the device based on graphical registration as in the second aspect. In the third aspect, the surgical robot can perform the method based on graphical registration by the device based on graphical registration, and the following effects can be achieved: improving the efficiency of determining the matching relationship between the coordinates in different coordinate systems.

[0022] In a fourth aspect, an electronic device is provided, including a processor and a memory, the memory being configured to store computer program code including computer instructions, and the electronic device being configured to execute the method as in the first aspect and any possible implementation manner thereof when the processor executes the computer instructions.

[0023] In a fifth aspect, another electronic device is provided, including a processor, a sending device, an input device, an output device, and a memory, the memory being configured to store computer program code including computer instructions, and the electronic device being configured to execute the method as in the first aspect and any possible implementation manner thereof when the processor executes the computer instructions.

[0024] In a sixth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program includes program instructions, and the program instructions, when executed by a processor, cause the processor to perform the method in the first aspect and any possible implementation manner thereof.

[0025] In a seventh aspect, a computer program product is provided, and the computer program product includes a computer program or instructions, and the computer program or instructions, when executed on a computer, cause the computer to perform the method in the first aspect and any possible implementation manner thereof.

[0026] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, rather than restrictive of the application.

[0027] In the embodiments of the present application, the n first coordinates are the coordinates of the n reference objects in the first coordinate system, and the n second coordinates are the coordinates of the n reference objects in the second coordinate system. After the registration device obtains the n first coordinates and the n second coordinates, the registration device determines a first coordinate set from the n first coordinates and a second coordinate set from the n second coordinates, wherein the first coordinate set includes m first coordinates of the n first coordinates, the second coordinate set includes m second coordinates of the n second coordinates, and m is an integer greater than or equal to 3 and less than or equal to n. Then, a first graph is obtained based on the coordinates in the first coordinate set, and a second graph is obtained based on the coordinates in the second coordinate set, wherein the shape of the first graph is the same as the shape of the second graph. Finally, a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set can be obtained based on the first graph and the second graph, thereby improving the efficiency of determining the matching relationship between the coordinates in different coordinate systems. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background art, the drawings needed to be used in the embodiments of the present application or the background art will be described below.

[0029] The drawings herein are incorporated into the specification and form part of the specification, and the drawings show embodiments consistent with the present application, and together with the specification, serve to explain the technical solutions of the present application.

[0030] Figure 1 A flowchart of a method for graphic registration provided in the embodiments of the present application is shown in the figure. Figure 2 A structural diagram of a device for graphic registration provided in the embodiments of the present application is shown in the figure. Figure 3 A hardware structural diagram of an electronic device provided in the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0031] In order to better understand the technical scheme of the present application, the technical scheme of the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0032] The terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0033] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood that the embodiments described herein are capable of combination. It will be understood that, in this application, "at least one", means one or more, "multiple" means two or more, and "at least two" means two or more than three.

[0034] The execution subject of the embodiments of the present application is a device based on graphic registration (hereinafter referred to as a registration device), wherein the registration device can be any electronic device capable of executing the technical solutions disclosed in the method embodiments of the present application. Alternatively, the registration device can be one of the following: a computer, a server.

[0035] It should be understood that the method embodiments of the present application can also be implemented by a processor executing computer program code. The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Please refer to Figure 1 , Figure 1 A flowchart of a method based on graphic registration provided by the embodiments of the present application.

[0036] 101、obtain n first coordinates and n second coordinates, wherein the n first coordinates are the coordinates of n reference objects in a first coordinate system, the n second coordinates are the coordinates of the n reference objects in a second coordinate system, and n is an integer greater than 3.

[0037] In some embodiments, the first coordinate system is an image coordinate system based on a CT image collected by a CT collection device, wherein the CT collection device is a device for collecting the CT image. The second coordinate system is a coordinate system established based on an optical tracking device, which can be used to determine the position of an optical marker in the second coordinate system.

[0038] The n reference objects can be any n objects. Optionally, the n first coordinates and the n second coordinates are the coordinates of the n objects in the first coordinate system and the second coordinate system at the same time, respectively.

[0039] It should be understood that although the n first coordinates are the coordinates of the n reference objects in the first coordinate system, in step 101, the correspondence between the n first coordinates and the n reference objects is unknown to the registration device. For example, the n reference objects include a reference object o1 and a reference object o2, and the n first coordinates include a first coordinate z1 and a first coordinate z2. The registration device cannot determine whether the first coordinate z1 is the coordinate of the reference object o1 in the first coordinate system or the first coordinate z2 is the coordinate of the reference object o1 in the first coordinate system based on the n first coordinates. Similarly, in step 101, the correspondence between the n second coordinates and the n reference objects is also unknown to the registration device.

[0040] 102. Determine a first coordinate set from the n first coordinates, wherein the first coordinate set includes m first coordinates from the n first coordinates, and m is an integer greater than or equal to 3 and less than or equal to n.

[0041] In some embodiments, the registration device selects m first coordinates from the n first coordinates, and obtains the first coordinate set based on the selected m first coordinates. Wherein m is an integer greater than or equal to 3 and less than or equal to n, i.e., m is an integer, and the minimum value of m is 3 and the maximum value of m is n.

[0042] 103. Determine a second coordinate set from the n second coordinates, wherein the second coordinate set includes m second coordinates from the n second coordinates.

[0043] In some embodiments, the registration device selects m second coordinates from the n second coordinates, and obtains the second coordinate set based on the selected m second coordinates.

[0044] 104. Obtain a first graph based on the coordinates in the first coordinate set.

[0045] One coordinate in the first coordinate set represents one point in the first coordinate system, and m coordinates in the first coordinate set represent m different points in the first coordinate system. The registration device obtains the first figure based on the coordinates in the first coordinate set, i.e., obtains the first figure based on the m points represented by the m coordinates in the first coordinate set.

[0046] In some schemes, m is 3, and the m coordinates in the first coordinate set represent 3 different points. The registration device can obtain a triangle based on the 3 different points, where the 3 different points are respectively three vertices of the triangle, and the first figure is the triangle.

[0047] In other schemes, m is 3, and the m coordinates in the first coordinate set represent 3 different points, and the 3 different points are not collinear. The registration device can obtain a circle based on the 3 different points, and the first figure is the circle.

[0048] In yet other schemes, m is 4, and the m coordinates in the first coordinate set represent 4 different points. The registration device can obtain a rectangle based on the 4 different points, where the 4 different points are respectively four vertices of the rectangle, and the first figure is the rectangle.

[0049] 105. Obtain a second figure based on the coordinates in the second coordinate set, where the shape of the first figure and the shape of the second figure are the same.

[0050] The registration device obtains the second figure in the same way as obtaining the first figure, so the shape of the first figure and the shape of the second figure are the same. For example, in the case where the first figure is a triangle, the second figure is also a triangle. In the case where the first figure is a rectangle, the second figure is also a rectangle.

[0051] Optionally, the first figure and the second figure are both closed figures. Since at least 3 points are needed to determine a closed figure, m is greater than or equal to 3, i.e., the coordinates in the first coordinate set and the coordinates in the second coordinate set are both at least 3.

[0052] 106. Obtain a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first figure and the second figure.

[0053] In the embodiments of the present application, the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set includes a matching coordinate pair in the first coordinate set and the second coordinate set, or the matching relationship includes that there is no matching coordinate pair in the first coordinate set and the second coordinate set. The matching coordinate pair includes one coordinate in the first coordinate set and one coordinate in the second coordinate set, and the two coordinates in the matching coordinate pair correspond to the same reference object in the n reference objects. For example, the first coordinate z1 in the first coordinate set is the coordinate of the reference object o1 in the first coordinate system, and the second coordinate z2 in the second coordinate set is the coordinate of the reference object o2 in the second coordinate system. At this time, the first coordinate z1 and the second coordinate z2 are a matching coordinate pair.

[0054] If a figure formed by the m reference objects corresponding to the m coordinates in the first coordinate set is referred to as a first reference figure, the similarity between the first figure and the first reference figure is high. If a figure formed by the m reference objects corresponding to the m coordinates in the second coordinate set is referred to as a second reference figure, the similarity between the second figure and the second reference figure is high. Therefore, if the first figure and the second figure are similar, it indicates that the first reference figure and the second reference figure are similar, that is, the probability that the m reference objects constituting the first reference figure and the m reference objects constituting the second reference figure are the same m reference objects is high, and the probability that the position of any one of the m reference objects in the first reference figure is the same as the position of the any one of the m reference objects in the second reference figure is high. Conversely, if the first figure and the second figure are not similar, it indicates that the first reference figure and the second reference figure are not similar, that is, the probability that the m reference objects constituting the first reference figure and the m reference objects constituting the second reference figure are different m reference objects is high. Therefore, the registration device can obtain the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first figure and the second figure.

[0055] In Figure 1In the shown method based on the graphic registration, the n first coordinates are the coordinates of the n references in the first coordinate system, and the n second coordinates are the coordinates of the n references in the second coordinate system. After obtaining the n first coordinates and the n second coordinates, the registration device determines a first coordinate set from the n first coordinates and a second coordinate set from the n second coordinates, wherein the first coordinate set includes m first coordinates from the n first coordinates, the second coordinate set includes m second coordinates from the n second coordinates, and m is an integer greater than or equal to 3 and less than or equal to n. Then, a first graphic is obtained based on the coordinates in the first coordinate set, and a second graphic is obtained based on the coordinates in the second coordinate set, wherein the shape of the first graphic is the same as the shape of the second graphic. Finally, a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set can be obtained based on the first graphic and the second graphic, thereby improving the efficiency of determining the matching relationship between the coordinates in different coordinate systems.

[0056] As an optional implementation, step 106 includes the following steps: 1061, determining an absolute value of a difference between the perimeters of the first graphic and the second graphic to obtain a first absolute value.

[0057] 1062, in the case where the first condition is met, determining that the matching relationship includes m groups of matching coordinate pairs.

[0058] The first condition includes that the first absolute value is less than or equal to a first threshold. The fact that the first condition is met indicates that the perimeters of the first graphic and the second graphic are relatively close, and thus the probability that the first graphic and the second graphic are similar is relatively high. At this time, m groups of matching coordinate pairs can be obtained based on the positions of the coordinates in the first coordinate set in the first graphic and the positions of the coordinates in the second coordinate set in the second graphic. Exemplarily, the first graphic and the second graphic are both triangles, the first coordinate set includes a first coordinate z1, a first coordinate z2, and a first coordinate z3, and the second coordinate set includes a second coordinate z4, a second coordinate z5, and a second coordinate z6. The first coordinate z1 corresponds to the largest internal angle in the first graphic, the first coordinate z2 corresponds to the second largest internal angle in the first graphic, and the first coordinate z3 corresponds to the smallest internal angle in the first graphic. The second coordinate z4 corresponds to the largest internal angle in the second graphic, the second coordinate z5 corresponds to the second largest internal angle in the second graphic, and the second coordinate z6 corresponds to the smallest internal angle in the second graphic. At this time, the first coordinate z1 and the second coordinate z4 can be determined as a group of matching coordinate pairs, the first coordinate z2 and the second coordinate z5 can be determined as a group of matching coordinate pairs, and the first coordinate z3 and the second coordinate z6 can be determined as a group of matching coordinate pairs.

[0059] 1063, in the case where the first condition is not met, determining that the matching relationship is that there is no matching coordinate pair between the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0060] The first condition not being met indicates that the perimeters of the first figure and the second figure are quite different, and that the probability of the first figure and the second figure being dissimilar is high, and it can be determined that there is no matching coordinate pair in the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0061] In this implementation, the registration device can determine whether the first figure and the second figure are similar based on the difference between the perimeter of the first figure and the perimeter of the second figure, and then determine the matching relationship of the coordinates in the first coordinate set and the coordinates in the second coordinate set, which can improve the efficiency of determining the matching relationship.

[0062] As an optional implementation, m is 3, and the shape of the first figure and the shape of the second figure are both triangles. The first condition further includes at least two of the following: the second absolute value is less than or equal to a second threshold value, the third absolute value is less than or equal to a third threshold value, and the fourth absolute value is less than or equal to a fourth threshold value. The second absolute value is the absolute value of the difference between the largest interior angle of the first figure and the largest interior angle of the second figure, the third absolute value is the absolute value of the difference between the second largest interior angle of the first figure and the second largest interior angle of the second figure, and the fourth absolute value is the absolute value of the difference between the smallest interior angle of the first figure and the smallest interior angle of the second figure.

[0063] In this implementation, the registration device can determine whether the first figure and the second figure are similar based on the difference between the perimeter of the first figure and the perimeter of the second figure, and the difference between the interior angles of the first figure and the interior angles of the second figure, thereby improving the accuracy of the determination. In this way, the accuracy of the matching relationship of the coordinates in the first coordinate set and the coordinates in the second coordinate set can be improved.

[0064] As an optional implementation, after obtaining the matching relationship of the coordinates in the first coordinate set and the coordinates in the second coordinate set, the registration device further performs the following steps: in the case where the matching relationship includes m groups of matching coordinate pairs, obtaining a first registration relationship between the first coordinate system and the second coordinate system based on the m groups of matching coordinate pairs, wherein the first registration relationship is used to convert coordinates in the first coordinate system into coordinates in the second coordinate system.

[0065] In some embodiments, the registration device obtains the first registration relationship by processing the m sets of matched coordinate pairs based on an iterative closest point (ICP) algorithm. Specifically, in the process of processing the m sets of matched coordinate pairs based on the ICP algorithm, multiple iterations are performed, and each iteration can obtain a first candidate registration relationship of the first coordinate system and the second coordinate system. Optionally, in the process of processing the m sets of matched coordinate pairs based on the ICP algorithm, if a second condition is met, the processing is stopped, wherein the second condition includes at least one of the following: a change rate of an error of the first candidate registration relationship obtained by adjacent two iterations is less than or equal to an error rate threshold, the error of the first candidate registration relationship is less than or equal to an error threshold, and a number of iterations based on the ICP algorithm is less than or equal to an iteration number threshold. The error of the first candidate registration relationship is a root mean square error (RMS) of n second coordinates and n third converted coordinates, wherein the third converted coordinates are converted from the first coordinates to the second coordinate system based on the first candidate registration relationship. After stopping the processing of the m sets of matched coordinate pairs based on the ICP algorithm, the first registration relationship can be obtained based on the multiple first candidate registration relationships.

[0066] Optionally, the registration device determines a first candidate registration relationship with the most registration points from the multiple first candidate registration relationships as the first registration relationship, wherein a registration point of the first candidate registration relationship refers to a difference between the first coordinate and the third converted coordinate being less than or equal to a registration threshold based on the first candidate registration relationship.

[0067] Optionally, n is greater than 3, and exemplarily, n = 6. The registration device determines at least one second candidate registration relationship with more than 3 registration points from the multiple first candidate registration relationships. In the case that the number of the second candidate registration relationships is 1, the second candidate registration relationship is determined as the first registration relationship. In the case that the number of the second candidate registration relationships is greater than 1, and the number of the second candidate registration relationship with the smallest error in the at least two second candidate registration relationships is 1, the second candidate registration relationship with the smallest error in the at least two second candidate registration relationships is determined as the first registration relationship. In the case that the number of the second candidate registration relationships is greater than 1, and the number of the second candidate registration relationship with the smallest error in the at least two second candidate registration relationships is greater than 1, the second candidate registration relationship with the smallest error and the smallest iteration number is determined as the first registration relationship, wherein the iteration number of the second candidate registration relationship refers to the number of completed iterations when the second candidate registration relationship is obtained.

[0068] In another solution, when m=3 and the first pattern and the second pattern are both triangles, the registration device processes the m sets of matching coordinate pairs based on a singular value decomposition (SVD) algorithm to obtain the first registration relationship.

[0069] Optionally, the registration device invokes multiple threads to process the m sets of matching coordinate pairs in parallel to obtain the first registration relationship, thereby improving the speed of obtaining the first registration relationship.

[0070] As an optional implementation, after obtaining the first registration relationship, the registration device further performs the following steps: converting the third coordinate based on the first registration relationship to obtain a fourth coordinate, wherein the third coordinate is one of the n first coordinates other than the coordinates in the first coordinate set; determining n-m second coordinates of the n second coordinates other than the coordinates in the second coordinate set; in a case where a fifth coordinate exists in the n-m second coordinates, obtaining a set of matching coordinate pairs, wherein the distance between the fifth coordinate and the third coordinate is less than or equal to the fifth threshold, and the set of matching coordinate pairs includes the third coordinate and the fifth coordinate; and obtaining a second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, wherein the second registration relationship is used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0071] In this implementation, after obtaining the first registration relationship, the registration device uses the third coordinate to further determine whether there is a matching coordinate pair other than the m sets of matching coordinate pairs in the first coordinate set and the second coordinate set based on the first registration relationship. Specifically, first, the registration device converts the third coordinate into a fourth coordinate in the second coordinate system based on the first registration relationship. If there is a coordinate matching the third coordinate in the n-m second coordinates, there should be a coordinate close to the fourth coordinate in the n-m second coordinates, which is the fifth coordinate described above. Therefore, the registration device determines that the third coordinate and the fifth coordinate are a set of matching coordinate pairs in a case where the fifth coordinate exists in the n-m second coordinates. Then, the registration device recalculates the registration relationship between the first coordinate system and the second coordinate system based on the m sets of matching coordinate pairs and the newly obtained set of matching coordinate pairs to obtain the second registration relationship.

[0072] Optionally, the fifth coordinate does not exist in the n-m second coordinates, indicating that the error of the first registration relationship is large, at this time, the matching coordinate pairs should be determined again from the n first coordinates and the n second coordinates. For example, m=3, the first pattern and the second pattern are both triangles. If at least two similar triangles can be obtained based on the n reference objects, then the 3 reference objects corresponding to the first pattern and the 3 reference objects corresponding to the second pattern can be different, but the similarity of the first pattern and the second pattern is high. At this time, it is easy to cause the error of the first registration relationship to be large. Therefore, the registration device determines a third coordinate set from the n first coordinates and a fourth coordinate set from the n second coordinates in the case that the fifth coordinate does not exist in the n-m second coordinates. A third pattern is obtained based on the third coordinate set, and a fourth pattern is obtained based on the fourth coordinate set. A matching relationship between the coordinates in the third coordinate set and the coordinates in the fourth coordinate set is obtained based on the third pattern and the fourth pattern. In the case that the matching relationship between the coordinates in the third coordinate set and the coordinates in the fourth coordinate set includes m groups of matching coordinate pairs (in order to distinguish the m groups of matching coordinate pairs in the first coordinate set and the second coordinate set, the m groups of matching coordinate pairs in the first coordinate set and the second coordinate set are referred to as new m groups of matching coordinate pairs below), a third registration relationship of the first coordinate system and the second coordinate system is obtained based on the new m groups of matching coordinate pairs. In this way, the accuracy of the registration relationship of the first coordinate system and the second coordinate system can be improved.

[0073] As an optional implementation, the first coordinate system is an image coordinate system of a first CT image, the first CT image is an image collected by a CT collection device at a first time, the second coordinate system is a coordinate system established based on an optical tracking device, the n reference objects are n optical markers attached to the surface of the first object, and the n second coordinates are determined by the optical tracking device at the first time.

[0074] In this implementation, the registration device further performs the following steps: converting the n first coordinates into n first converted coordinates in the second coordinate system based on the second registration relationship. A first error of the second registration relationship is obtained based on the difference between the n first converted coordinates and the n second coordinates, wherein the difference is positively correlated with the first error. A second error of the reference registration relationship is obtained, wherein the reference registration relationship is used to convert coordinates in an image coordinate system of a second CT image into coordinates in the second coordinate system, the second CT image is collected by the CT collection device at a second time, the reference registration relationship is obtained based on the positions of the n optical markers in the image coordinate system of the second CT image and n sixth coordinates of the n optical markers in the second coordinate system, and the n sixth coordinates are determined by the optical tracking device at the second time. The difference between the first error and the second error is used to determine the difference between the breathing state of the first object at the first time and the breathing state of the first object at the second time.

[0075] In this implementation, the respiration of the first object causes the body surface of the first object to move, which in turn causes the respiration state at different time instants to be different, thereby causing the position of the optical marker in the body surface to change. The registration relationship between the image coordinate system of the CT image and the second coordinate system is obtained based on the position of the optical marker. Therefore, the registration relationship obtained at different time instants corresponding to different respiration states is different. Conversely, the difference in the error of the two registration relationships can be used to determine the difference in the respiration state of the time instants of the two registration relationships.

[0076] In a possible implementation scenario, in the medical field, it is often necessary to plan a target path from the skin region of the first object to the tissue in the first object. The CT image obtained by scanning and collecting the first object by the CT collection device includes information of the skin region of the first object and information of the tissue (such as the liver) in the first object. Therefore, the target path can be planned based on the CT image of the first object. Specifically, the target path can be planned based on the second CT image. After the target path is obtained, the target object can be controlled to move along the target path from the skin region of the first object to the tissue in the first object based on the mechanical arm. Exemplarily, the target object is a needle.

[0077] Since the coordinate system established based on the optical tracking device is the same as the world coordinate system, that is, the second coordinate system is the same as the world coordinate system, converting the target path in the CT image into the second coordinate system is beneficial to controlling the target object to move along the target path by the mechanical arm and facilitating the doctor to observe the relative positional relationship between the target path and the first object. Therefore, the reference registration relationship needs to be obtained based on the positions of the n optical markers in the image coordinate system of the second CT image and the n sixth coordinates.

[0078] Considering that the respiration of the first object causes the body surface of the first object and the tissue in the first object to move, which in turn causes the respiration state at different time instants to be different, thereby causing the path from the skin region of the body surface to the tissue in the body to be different at different time instants. Therefore, in the case that the second CT image is collected at the second time instant and the n sixth coordinates are determined at the second time instant, the time instant at which the respiration state is close to the respiration state at the second time instant should be determined as the time instant at which the target object is controlled to move along the target path from the skin region of the first object to the tissue in the first object (hereinafter referred to as the moving time instant). In this way, the success rate of moving the target object along the target path from the skin region of the first object to the tissue in the first object can be improved.

[0079] According to the embodiment, the difference of the respiratory state at the time when the two registration relationships are determined can be determined, and thus the moving time can be determined. Optionally, the registration relationship between the image coordinate system of the CT image obtained at the moving time and the second coordinate system is referred to as a moving registration relationship. The difference between the error of the moving registration relationship and the error of the reference registration relationship is less than or equal to an eighth threshold value.

[0080] In some schemes, after the target path is obtained, the CT acquisition device periodically acquires CT images of the first object, and the registration device determines the coordinates of the n optical markers in the image coordinate system of each CT image, and the optical tracking device periodically determines the coordinates of the n optical markers in the second coordinate system. Then, the moving time can be determined based on all the CT images and the coordinates determined by the optical tracking device.

[0081] Optionally, because the respiration of the first object is periodic, in order to reduce the storage overhead of the CT images acquired by the CT acquisition device and the coordinates determined by the optical tracking device, the registration device stores the CT images acquired at a time point within a preset time period from the current time point and the coordinates determined at a time point within the preset time period from the current time point. The preset time period is the duration of the respiration cycle of the first object. For example, the preset time period is 5 seconds.

[0082] As an optional embodiment, before performing the step of "obtaining the second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matched coordinate pairs and the set of matched coordinate pairs", the registration device further performs the following steps: obtaining a seventh coordinate of a first optical marker in the n optical markers, wherein the seventh coordinate is determined by the optical tracking device at a third time point, the time difference between the first time point and the third time point is less than a sixth threshold value, and the sixth threshold value is based on the duration of the respiration cycle of the first object; determining an eighth coordinate from the n second coordinates, wherein the eighth coordinate is the coordinate of the first optical marker in the second coordinate system; determining the absolute value of the difference between the seventh coordinate and the eighth coordinate to obtain a fifth absolute value; obtaining a constraint condition for the registration relationship between the first coordinate system and the second coordinate system based on the fifth absolute value, wherein the constraint condition includes that the absolute value of the difference between the second conversion coordinate and the eighth coordinate is less than or equal to a seventh threshold value, the second conversion coordinate is obtained by converting the seventh coordinate to the coordinate in the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system, and the seventh threshold value is positively correlated with the fifth absolute value. After obtaining the constraint condition, the following steps are performed during the execution of the step of "obtaining the second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matched coordinate pairs and the set of matched coordinate pairs": obtaining the second registration relationship based on the constraint condition, the m sets of matched coordinate pairs and the set of matched coordinate pairs.

[0083] In the implementation, the smaller the fifth absolute value is, the smaller the difference between the positions of the first optical marker at different times is, i.e., the smaller the influence of the respiration of the first object on the first optical marker is, and thus the higher the stability of the coordinates of the first optical marker is. In order to improve the accuracy of the registration relationship between the first coordinate system and the second coordinate system, the registration relationship can be constrained based on the seventh coordinates of the first optical marker in the first coordinate system and the eighth coordinates of the first optical marker in the second coordinate system. Specifically, the coordinates obtained by converting the seventh coordinates into the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system are second converted coordinates. The absolute value of the difference between the second converted coordinates and the eighth coordinates represents the conversion error of the registration relationship on the coordinates of the first optical marker. The higher the stability of the coordinates of the first optical marker is, the smaller the conversion error should be, which can effectively improve the accuracy of the registration relationship. Therefore, the constraint condition includes that the absolute value of the difference between the second converted coordinates and the eighth coordinates is less than or equal to a seventh threshold value, where the seventh threshold value is positively correlated with the fifth absolute value.

[0084] After obtaining the constraint condition, the second registration relationship can be obtained based on the constraint condition, the m sets of matching coordinate pairs and the one set of matching coordinate pairs, which can improve the accuracy of the second registration relationship. Specifically, the registration device obtains the second registration relationship that satisfies the constraint condition based on the m sets of matching coordinate pairs and the one set of matching coordinate pairs.

[0085] In some schemes, the seventh threshold value is further related to at least one of the following: the confidence of the matching coordinate pair corresponding to the first optical marker, and the probability that the first optical marker is blocked, where the seventh threshold value is negatively correlated with the confidence of the matching coordinate pair corresponding to the first optical marker, and the seventh threshold value is positively correlated with the probability that the first optical marker is blocked. Specifically, the matching coordinate pair corresponding to the first optical marker is the matching coordinate pair including the seventh coordinates and the eighth coordinates in the m sets of matching coordinate pairs. The confidence of the matching coordinate pair corresponding to the first optical marker is negatively correlated with the first absolute value. The higher the confidence of the matching coordinate pair corresponding to the first optical marker is, the smaller the above-mentioned conversion error should be, which can effectively improve the accuracy of the registration relationship. The first optical marker being blocked means that there is an occlusion between the optical tracking device and the first optical marker, and the existence of the occlusion will cause a large error in the eighth coordinates determined by the optical tracking device.

[0086] Optionally, represents the seventh threshold value, represents the stability of the coordinates of the first optical marker, represents the confidence of the matching coordinate pair corresponding to the first optical marker, represents the probability that the first optical marker is blocked, and thus , , represents that the following formula is satisfied: Equation (1) Optionally, is calculated by the following equation: Equation (2) Wherein, represents the fifth absolute value.

[0087] Those skilled in the art can understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0088] If the technical solution of the present application involves personal information, the product applying the technical solution of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solution of the present application involves sensitive personal information, the product applying the technical solution of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent sign is set to inform that the personal information collection range has been entered, and the personal information will be collected. If the individual voluntarily enters the collection range, it is considered to agree to collect the personal information; or on the device for processing personal information, the personal information processing rules are informed by using obvious signs / information, and the personal authorization is obtained by pop-up information or asking the individual to upload his / her personal information, etc. Wherein, the personal information processing can include personal information processor, personal information processing purpose, processing method and personal information type, etc.

[0089] The above describes the method of the embodiment of the present application in detail, and the device of the embodiment of the present application is provided below.

[0090] Please refer to Figure 2 , Figure 2 A structure diagram of a device based on graphic registration provided by the embodiment of the present application. The device based on graphic registration 1 comprises an acquisition unit 11 and a processing unit 12, specifically: The acquisition unit 11 is configured to acquire n first coordinates and n second coordinates, the n first coordinates being coordinates of n reference objects in a first coordinate system, the n second coordinates being coordinates of the n reference objects in a second coordinate system, and the n being an integer greater than 3; The processing unit 12 is configured to determine a first coordinate set from the n first coordinates, the first coordinate set comprising m first coordinates in the n first coordinates, and the m being an integer greater than or equal to 3 and less than or equal to the n. The processing unit 12 is further configured to determine a second coordinate set from the n second coordinates, the second coordinate set including m second coordinates of the n second coordinates. The processing unit 12 is further configured to obtain a first graph based on the coordinates in the first coordinate set. The processing unit 12 is further configured to obtain a second graph based on the coordinates in the second coordinate set, the first graph and the second graph having the same shape. The processing unit 12 is further configured to obtain a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first graph and the second graph.

[0091] In combination with any of the embodiments of the present application, the processing unit 12 is further configured to: determine an absolute value of a difference between a perimeter of the first graph and a perimeter of the second graph to obtain a first absolute value; in a case where a first condition is met, determine that the matching relationship includes m groups of matching coordinate pairs; wherein the first condition includes that the first absolute value is less than or equal to a first threshold value; the matching coordinate pair includes one coordinate in the first coordinate set and one coordinate in the second coordinate set, and the two coordinates in the matching coordinate pair correspond to a same one of the n reference objects; and the m groups of matching coordinate pairs are obtained based on positions of the coordinates in the first coordinate set in the first graph and positions of the coordinates in the second coordinate set in the second graph. in a case where the first condition is not met, determine that the matching relationship is that there is no matching coordinate pair between the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0092] In combination with any of the embodiments of the present application, the m is 3, and the shape of the first graph and the shape of the second graph are both triangles. The first condition further includes at least two of the following: a second absolute value is less than or equal to a second threshold value, a third absolute value is less than or equal to a third threshold value, and a fourth absolute value is less than or equal to a fourth threshold value. wherein the second absolute value is an absolute value of a difference between a largest interior angle in the first graph and a largest interior angle in the second graph, the third absolute value is an absolute value of a difference between a second largest interior angle in the first graph and a second largest interior angle in the second graph, and the fourth absolute value is an absolute value of a difference between a smallest interior angle in the first graph and a smallest interior angle in the second graph.

[0093] In combination with any of the embodiments of the present application, the processing unit 12 is further configured to: In a case where the matching relationship includes the m sets of matching coordinate pairs, a first registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matching coordinate pairs, and the first registration relationship is used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0094] In combination with any of the embodiments of the present application, the m is less than the n, and the processing unit 12 is further configured to: convert a third coordinate into a fourth coordinate based on the first registration relationship, the third coordinate being one of the n first coordinates other than the coordinates in the first coordinate set; determine n-m second coordinates of the n second coordinates other than the coordinates in the second coordinate set; in a case where a fifth coordinate exists in the n-m second coordinates, the distance between the fifth coordinate and the third coordinate being less than or equal to a fifth threshold, obtain a set of matching coordinate pairs, the set of matching coordinate pairs including the third coordinate and the fifth coordinate; obtain a second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, and the second registration relationship is used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0095] In combination with any of the embodiments of the present application, the first coordinate system is an image coordinate system of a first CT image, the first CT image being an image collected by a CT collection device at a first time, the second coordinate system is a coordinate system established based on an optical tracking device, the n reference objects are n optical markers attached to a body surface of a first object, and the n second coordinates are determined by the optical tracking device at the first time. The processing unit 12 is further configured to convert the n first coordinates into n first converted coordinates in the second coordinate system based on the second registration relationship. The processing unit 12 is further configured to obtain a first error of the second registration relationship based on a difference between the n first converted coordinates and the n second coordinates, the difference being positively correlated with the first error. The acquisition unit 11 is further configured to obtain a second error of a reference registration relationship, the reference registration relationship being used to convert a coordinate in an image coordinate system of a second CT image into a coordinate in the second coordinate system, the second CT image being collected by the CT collection device at a second time, the reference registration relationship being obtained based on positions of the n optical markers in the image coordinate system of the second CT image and n sixth coordinates of the n optical markers in the second coordinate system, the n sixth coordinates being determined by the optical tracking device at the second time. The processing unit 12 is further configured to determine a difference between the respiratory state of the first object at the first time and the respiratory state of the first object at the second time based on a difference between the first error and the second error.

[0096] In combination with any of the embodiments of the present application, the acquisition unit 11 is further configured to acquire a seventh coordinate of a first optical marker in the n optical markers, the seventh coordinate being determined by the optical tracking device at a third time, a time difference between the first time and the third time being less than a sixth threshold value, the sixth threshold value being based on a duration of a respiratory cycle of the first object. The processing unit 12 is further configured to determine an eighth coordinate from the n second coordinates, the eighth coordinate being a coordinate of the first optical marker in the second coordinate system. The processing unit 12 is further configured to determine an absolute value of a difference between the seventh coordinate and the eighth coordinate, obtaining a fifth absolute value. The processing unit 12 is further configured to obtain, based on the fifth absolute value, a constraint condition that the constraint condition is a registration relationship between the first coordinate system and the second coordinate system, the constraint condition including that an absolute value of a difference between a second conversion coordinate and the eighth coordinate is less than or equal to a seventh threshold value, the second conversion coordinate being obtained by converting the seventh coordinate into a coordinate in the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system, the seventh threshold value being positively correlated with the fifth absolute value. The processing unit 12 is further configured to obtain the second registration relationship based on the constraint condition, the m sets of matching coordinate pairs, and the set of matching coordinate pairs.

[0097] In the embodiments of the present application, the n first coordinates are coordinates of n reference objects in a first coordinate system, and the n second coordinates are coordinates of the n reference objects in a second coordinate system. After the registration device acquires the n first coordinates and the n second coordinates, the registration device determines a first coordinate set from the n first coordinates and a second coordinate set from the n second coordinates, wherein the first coordinate set includes m first coordinates in the n first coordinates, the second coordinate set includes m second coordinates in the n second coordinates, and m is an integer greater than or equal to 3 and less than or equal to n. Then, a first graph is obtained based on the coordinates in the first coordinate set, and a second graph is obtained based on the coordinates in the second coordinate set, wherein the shape of the first graph is the same as the shape of the second graph. Finally, a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set can be obtained based on the first graph and the second graph, thereby improving the efficiency of determining the matching relationship between the coordinates in different coordinate systems.

[0098] In some embodiments, the apparatus provided by the embodiments of the present application has functions or includes modules for performing the methods described in the above method embodiments, and the specific implementation can be referred to the description of the above method embodiments. For the purpose of brevity, details are not described here again.

[0099] Figure 3 A hardware structure schematic diagram of an electronic device provided by the embodiments of the present application is shown in FIG. 2. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 further includes an input device 23 and an output device 24. The processor 21, the memory 22, the input device 23 and the output device 24 are coupled through a connector, which includes various interfaces, transmission lines or buses, etc., and the embodiments of the present application do not make any limitation. It should be understood that in various embodiments of the present application, coupling means mutual contact in a specific way, including direct connection or indirect connection through other devices, for example, various interfaces, transmission lines, buses, etc.

[0100] The processor 21 can be one or more graphics processing units (GPUs). In the case that the processor 21 is a GPU, the GPU can be a single-core GPU or a multi-core GPU. Optionally, the processor 21 can be a processor group composed of multiple GPUs, and the multiple processors are coupled with each other through one or more buses. Optionally, the processor can also be other types of processors, etc., and the embodiments of the present application do not make any limitation.

[0101] The memory 22 can be used to store computer program instructions and various computer program codes for executing the schemes of the present application. Optionally, the memory includes but is not limited to a random access memory (RAM), a read-only memory (ROM), an erasable programmable read only memory (EPROM), or a compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0102] The input device 23 is used for inputting data and / or signals, and the output device 24 is used for outputting data and / or signals. The input device 23 and the output device 24 can be independent devices, or can be an integral device.

[0103] It can be understood that in the embodiments of the present application, the memory 22 can be used not only for storing related instructions, but also for storing related data, and the embodiments of the present application do not make any limitation on the data stored in the memory.

[0104] It can be understood that, Figure 3 Only a simplified design of an electronic device is shown. In actual applications, the electronic device can also include other necessary elements, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the protection scope of the present application.

[0105] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0106] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here. Those skilled in the art can also clearly understand that each embodiment of the present application describes each with emphasis, and for the convenience and brevity of the description, the same or similar parts in different embodiments can not be described in detail, so the parts not described or not described in detail in an embodiment can be referred to the description of other embodiments.

[0107] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0108] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0109] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0110] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in or transmitted by a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital versatile disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0111] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be instructed by a computer program to complete the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium includes a read-only memory (ROM) or a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

Claims

1. A method based on image registration, characterized in that, The method includes: Obtain n first coordinates and n second coordinates, wherein the n first coordinates are the coordinates of n reference objects in the first coordinate system, and the n second coordinates are the coordinates of the n reference objects in the second coordinate system, wherein n is an integer greater than 3; A first coordinate set is determined from the n first coordinates, the first coordinate set including m first coordinates from the n first coordinates, where m is an integer greater than or equal to 3 and less than or equal to n; A second coordinate set is determined from the n second coordinates, wherein the second coordinate set includes m second coordinates from the n second coordinates; The first graphic is obtained based on the coordinates in the first coordinate set; The second graphic is obtained based on the coordinates in the second coordinate set, and the shape of the first graphic is the same as the shape of the second graphic. Based on the first and second graphics, the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set is obtained.

2. The method according to claim 1, characterized in that, The step of obtaining the matching relationship between coordinates in the first coordinate set and coordinates in the second coordinate set based on the first and second graphics includes: Determine the absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure to obtain the first absolute value; If the first condition is met, the matching relationship is determined to include m sets of matching coordinate pairs; Wherein, the first condition includes the first absolute value being less than or equal to a first threshold; the matching coordinate pair includes a coordinate from the first coordinate set and a coordinate from the second coordinate set, and the two coordinates in the matching coordinate pair correspond to the same reference object among the n reference objects; the m sets of matching coordinate pairs are obtained based on the position of the coordinate in the first coordinate set in the first graphic and the position of the coordinate in the second coordinate set in the second graphic; If the first condition is not met, the matching relationship is determined to be that there is no matching coordinate pair in the coordinates of the first coordinate set and the coordinates of the second coordinate set.

3. The method according to claim 2, characterized in that, The value of m is 3, and the shapes of both the first and second figures are triangles. The first condition also includes at least two of the following: the second absolute value is less than or equal to the second threshold, the third absolute value is less than or equal to the third threshold, and the fourth absolute value is less than or equal to the fourth threshold; Wherein, the second absolute value is the absolute value of the difference between the largest interior angle in the first figure and the largest interior angle in the second figure, the third absolute value is the absolute value of the difference between the second largest interior angle in the first figure and the second largest interior angle in the second figure, and the fourth absolute value is the absolute value of the difference between the smallest interior angle in the first figure and the smallest interior angle in the second figure.

4. The method according to claim 2 or 3, characterized in that, The method further includes: When the matching relationship includes the m sets of matching coordinate pairs, a first registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matching coordinate pairs. The first registration relationship is used to convert the coordinates in the first coordinate system into the coordinates in the second coordinate system.

5. The method according to claim 4, characterized in that, Where m is less than n, the method further includes: Based on the first registration relationship, the third coordinate is transformed to obtain the fourth coordinate, wherein the third coordinate is one of the n first coordinates other than the coordinates in the first coordinate set; Determine nm of the n second coordinates, excluding the coordinates in the second coordinate set; If a fifth coordinate exists among the nm second coordinates, a set of matching coordinate pairs is obtained, wherein the distance between the fifth coordinate and the third coordinate is less than or equal to a fifth threshold, and the set of matching coordinate pairs includes the third coordinate and the fifth coordinate; Based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, a second registration relationship between the first coordinate system and the second coordinate system is obtained. The second registration relationship is used to convert the coordinates in the first coordinate system into the coordinates in the second coordinate system.

6. The method according to claim 5, characterized in that, The first coordinate system is the image coordinate system of the first CT image, which is the image acquired by the CT acquisition device at the first moment. The second coordinate system is a coordinate system established based on the optical tracking device. The n reference objects are n optical markers attached to the surface of the first object. The n second coordinates are determined by the optical tracking device at the first moment. The method further includes: Based on the second registration relationship, the n first coordinates are converted into n first transformed coordinates in the second coordinate system; Based on the difference between the n first transformation coordinates and the n second coordinates, a first error of the second registration relationship is obtained, and the difference is positively correlated with the first error; The second error of the reference registration relationship is obtained. The reference registration relationship is used to convert the coordinates in the image coordinate system of the second CT image into coordinates in the second coordinate system. The second CT image is acquired by the CT acquisition device at the second time. The reference registration relationship is based on the positions of the n optical markers in the image coordinate system of the second CT image and the n sixth coordinates of the n optical markers in the second coordinate system. The n sixth coordinates are determined by the optical tracking device at the second time. Based on the difference between the first error and the second error, the difference between the breathing state of the first object at the first time and the breathing state of the first object at the second time is determined.

7. The method according to claim 6, characterized in that, Before obtaining the second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, the method further includes: The seventh coordinate of the first optical marker among the n optical markers is obtained. The seventh coordinate is determined by the optical tracking device at a third time. The time difference between the first time and the third time is less than a sixth threshold. The sixth threshold is based on the duration of the respiratory cycle of the first object. An eighth coordinate is determined from the n second coordinates, wherein the eighth coordinate is the coordinate of the first optical marker in the second coordinate system; Determine the absolute value of the difference between the seventh coordinate and the eighth coordinate to obtain the fifth absolute value; Based on the fifth absolute value, the constraint conditions for the registration relationship between the first coordinate system and the second coordinate system are obtained. The constraint conditions include that the absolute value of the difference between the second transformed coordinate and the eighth coordinate is less than or equal to the seventh threshold. The second transformed coordinate is obtained by converting the seventh coordinate into coordinates in the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system. The seventh threshold is positively correlated with the fifth absolute value. The step of obtaining the second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matching coordinate pairs and the first set of matching coordinate pairs includes: Based on the constraints, the m sets of matching coordinate pairs, and the set of matching coordinate pairs, the second registration relationship is obtained.

8. A device based on image registration, characterized in that, The graph-based registration device includes: An acquisition unit is used to acquire n first coordinates and n second coordinates, wherein the n first coordinates are the coordinates of n reference objects in a first coordinate system, and the n second coordinates are the coordinates of the n reference objects in a second coordinate system, wherein n is an integer greater than 3; A processing unit is configured to determine a first coordinate set from the n first coordinates, wherein the first coordinate set includes m first coordinates from the n first coordinates, and m is an integer greater than or equal to 3 and less than or equal to n; The processing unit is further configured to determine a second coordinate set from the n second coordinates, wherein the second coordinate set includes m second coordinates from the n second coordinates; The processing unit is further configured to obtain a first graphic based on the coordinates in the first coordinate set; The processing unit is further configured to obtain a second graphic based on the coordinates in the second coordinate set, wherein the shape of the first graphic is the same as the shape of the second graphic; The processing unit is further configured to obtain a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first graphic and the second graphic.

9. A surgical robot, characterized in that, Includes the graphic registration-based apparatus as described in claim 8.

10. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1 to 7.

Images