Method, surgical robot and related products based on image registration

Abstract

The application discloses a method based on image registration, a surgical robot and related products. The method comprises the following steps: obtaining n first coordinates and n second coordinates, wherein the n first coordinates are the coordinates of n reference objects in a first coordinate system, and the n second coordinates are the coordinates of the n reference objects in a second coordinate system. A first coordinate set is determined from the n first coordinates. A second coordinate set is determined from the n second coordinates. A first image is obtained based on the coordinates in the first coordinate set. A second image is obtained based on the coordinates in the second coordinate set, and the shapes of the first image and the second image are the same. A matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set is obtained based on the first image and the second image. The application also discloses corresponding products. The method provided by the application can improve the efficiency of determining the matching relationship between the coordinates in different coordinate systems.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a method based on graphic registration, a surgical robot and related products. BACKGROUND

[0002] In the medical field, it is often necessary to convert coordinates in different coordinate systems. Therefore, it is necessary to determine the registration relationship between different coordinate systems, wherein the registration relationship is used to convert coordinates in one coordinate system to coordinates in another coordinate system.

[0003] The current technology usually determines the matching relationship between the coordinates in different coordinate systems first, and then calculates the registration relationship between the different coordinate systems based on the matching relationship.

[0004] However, the efficiency of determining the matching relationship between the coordinates in different coordinate systems by the current technology is low. SUMMARY

[0005] The present application provides a method based on graphic registration, a surgical robot and related products, wherein the related products include a device based on graphic registration and an electronic device, so as to improve the efficiency of determining the matching relationship between the coordinates in different coordinate systems.

[0006] In a first aspect, a method based on graphic registration is provided, and the method comprises:

[0007] n first coordinates and n second coordinates are obtained, the n first coordinates are coordinates of n reference objects in a first coordinate system, the n second coordinates are coordinates of the n reference objects in a second coordinate system, and n is an integer greater than 3;

[0008] A first coordinate set is determined from the n first coordinates, the first coordinate set includes m first coordinates in the n first coordinates, and m is an integer greater than or equal to 3 and less than or equal to n;

[0009] A second coordinate set is determined from the n second coordinates, the second coordinate set includes m second coordinates in the n second coordinates;

[0010] A first graph is obtained based on the coordinates in the first coordinate set;

[0011] A second graph is obtained based on the coordinates in the second coordinate set, and the shape of the first graph is the same as the shape of the second graph;

[0012] Based on the first graph and the second graph, a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set is obtained.

[0013] In combination with any of the embodiments of the present application, the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set is obtained based on the first graph and the second graph, including:

[0014] An absolute value of a difference between the perimeter of the first graph and the perimeter of the second graph is determined to obtain a first absolute value;

[0015] In a case where the first condition is met, the matching relationship includes m groups of matching coordinate pairs;

[0016] The first condition includes that the first absolute value is less than or equal to a first threshold value; the matching coordinate pair includes one coordinate in the first coordinate set and one coordinate in the second coordinate set, and the two coordinates in the matching coordinate pair correspond to the same one of the n reference objects; and the m groups of matching coordinate pairs are obtained based on the positions of the coordinates in the first coordinate set in the first graph and the positions of the coordinates in the second coordinate set in the second graph.

[0017] In a case where the first condition is not met, the matching relationship is that there is no matching coordinate pair in the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0018] In combination with any of the embodiments of the present application, the m is 3, and the shapes of the first graph and the second graph are both triangles.

[0019] The first condition further includes at least two of the following: a second absolute value is less than or equal to a second threshold value, a third absolute value is less than or equal to a third threshold value, and a fourth absolute value is less than or equal to a fourth threshold value.

[0020] The second absolute value is an absolute value of a difference between the largest interior angle in the first graph and the largest interior angle in the second graph, the third absolute value is an absolute value of a difference between the second largest interior angle in the first graph and the second largest interior angle in the second graph, and the fourth absolute value is an absolute value of a difference between the smallest interior angle in the first graph and the smallest interior angle in the second graph.

[0021] In this embodiment, if two triangles are similar, the difference between the perimeters of the two triangles should be less than or equal to the first threshold value, the difference between the largest interior angles of the two triangles should be less than or equal to the second threshold value, the difference between the second largest interior angles of the two triangles should be less than or equal to the third threshold value, and the difference between the smallest interior angles of the two triangles should be less than or equal to the fourth threshold value. Therefore, the accuracy of determining whether the two first graphs and the second graph are similar based on the first condition can be improved, and the accuracy of the matching relationship can be improved.

[0022] In combination with any of the embodiments of the present application, the method further includes:

[0023] In a case where the matching relationship includes the m sets of matching coordinate pairs, a first registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matching coordinate pairs, and the first registration relationship is used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0024] In combination with any of the embodiments of the present application, the m is less than the n, and the method further includes:

[0025] A fourth coordinate is obtained by converting a third coordinate based on the first registration relationship, the third coordinate being one of the n first coordinates other than the coordinates in the first coordinate set;

[0026] n-m second coordinates other than the coordinates in the second coordinate set are determined from the n second coordinates;

[0027] In a case where a fifth coordinate exists in the n-m second coordinates, the distance between the fifth coordinate and the third coordinate is less than or equal to a fifth threshold value, a set of matching coordinate pairs is obtained, and the set of matching coordinate pairs includes the third coordinate and the fifth coordinate;

[0028] A second registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, and the second registration relationship is used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0029] In combination with any of the embodiments of the present application, the first coordinate system is an image coordinate system of a first computed tomography (CT) image, the first CT image is an image collected by a CT collection device at a first time, the second coordinate system is a coordinate system established based on an optical tracking device, the n reference objects are n optical markers attached to a body surface of a first object, and the n second coordinates are determined by the optical tracking device at the first time.

[0030] The method further includes:

[0031] The n first coordinates are converted into n first converted coordinates in the second coordinate system based on the second registration relationship;

[0032] A first error of the second registration relationship is obtained based on a difference between the n first converted coordinates and the n second coordinates, and the difference is positively correlated with the first error.

[0033] obtain a second error of a reference registration relationship, the reference registration relationship being used for converting a coordinate in an image coordinate system of a second CT image into a coordinate in the second coordinate system, the second CT image being collected by the CT collection device at a second time, the reference registration relationship being obtained based on positions of the n optical markers in the image coordinate system of the second CT image and n sixth coordinates of the n optical markers in the second coordinate system, the n sixth coordinates being determined by the optical tracking device at the second time;

[0034] determine a difference between a respiratory state of the first object at the first time and a respiratory state of the first object at the second time based on a difference between the first error and the second error.

[0035] In combination with any of the embodiments of the present application, before the second registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matched coordinate pairs and the set of matched coordinate pairs, the method further comprises:

[0036] obtain a seventh coordinate of a first optical marker in the n optical markers, the seventh coordinate being determined by the optical tracking device at a third time, a time difference between the first time and the third time being less than a sixth threshold value, the sixth threshold value being based on a duration of a respiratory cycle of the first object;

[0037] determine an eighth coordinate from the n second coordinates, the eighth coordinate being a coordinate of the first optical marker in the second coordinate system;

[0038] determine an absolute value of a difference between the seventh coordinate and the eighth coordinate to obtain a fifth absolute value;

[0039] obtain the constraint condition as a constraint condition of a registration relationship between the first coordinate system and the second coordinate system based on the fifth absolute value, the constraint condition comprising that an absolute value of a difference between a second conversion coordinate and the eighth coordinate is less than or equal to a seventh threshold value, the second conversion coordinate being obtained by converting the seventh coordinate into a coordinate in the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system, the seventh threshold value being positively correlated with the fifth absolute value;

[0040] The second registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matched coordinate pairs and the set of matched coordinate pairs, comprising:

[0041] The second registration relationship is obtained based on the constraint condition, the m sets of matched coordinate pairs and the set of matched coordinate pairs.

[0042] In a second aspect, a device for image registration is provided. The device for image registration comprises:

[0043] an obtaining unit, configured to obtain n first coordinates and n second coordinates, the n first coordinates being coordinates of n reference objects in a first coordinate system, the n second coordinates being coordinates of the n reference objects in a second coordinate system, n being an integer greater than 3;

[0044] a processing unit, configured to determine a first coordinate set from the n first coordinates, the first coordinate set comprising m first coordinates in the n first coordinates, m being an integer greater than or equal to 3 and less than or equal to n;

[0045] the processing unit is further configured to determine a second coordinate set from the n second coordinates, the second coordinate set comprising m second coordinates in the n second coordinates;

[0046] the processing unit is further configured to obtain a first image based on the coordinates in the first coordinate set;

[0047] the processing unit is further configured to obtain a second image based on the coordinates in the second coordinate set, the first image having a same shape as the second image;

[0048] the processing unit is further configured to obtain a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first image and the second image.

[0049] In combination with any of the embodiments of the present application, the processing unit is further configured to:

[0050] determine an absolute value of a difference between a perimeter of the first image and a perimeter of the second image to obtain a first absolute value;

[0051] in a case where a first condition is met, the matching relationship comprises m matching coordinate pairs;

[0052] wherein the first condition comprises that the first absolute value is less than or equal to a first threshold value, the matching coordinate pair comprises one coordinate in the first coordinate set and one coordinate in the second coordinate set, and the two coordinates in the matching coordinate pair correspond to a same one of the n reference objects, and the m matching coordinate pairs are obtained based on positions of the coordinates in the first coordinate set in the first image and positions of the coordinates in the second coordinate set in the second image;

[0053] in a case where the first condition is not met, the matching relationship is that the matching coordinate pair does not exist in the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0054] In any of the embodiments of the present application, m is 3, and the shape of the first pattern and the shape of the second pattern are both triangles.

[0055] The first condition further includes at least two of the following: a second absolute value is less than or equal to a second threshold value, a third absolute value is less than or equal to a third threshold value, and a fourth absolute value is less than or equal to a fourth threshold value.

[0056] The second absolute value is an absolute value of a difference between a largest internal angle of the first pattern and a largest internal angle of the second pattern, the third absolute value is an absolute value of a difference between a second largest internal angle of the first pattern and a second largest internal angle of the second pattern, and the fourth absolute value is an absolute value of a difference between a smallest internal angle of the first pattern and a smallest internal angle of the second pattern.

[0057] In any of the embodiments of the present application, the processing unit is further configured to:

[0058] In a case where the matching relationship includes the m sets of matching coordinate pairs, a first registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matching coordinate pairs, and the first registration relationship is used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0059] In any of the embodiments of the present application, m is less than n, and the processing unit is further configured to:

[0060] A fourth coordinate is obtained by converting a third coordinate based on the first registration relationship, the third coordinate being one of the n first coordinates other than the coordinates in the first coordinate set;

[0061] n-m second coordinates other than the coordinates in the second coordinate set are determined from the n second coordinates.

[0062] In a case where a fifth coordinate exists in the n-m second coordinates, a set of matching coordinate pairs is obtained, the distance between the fifth coordinate and the third coordinate is less than or equal to a fifth threshold value, and the set of matching coordinate pairs includes the third coordinate and the fifth coordinate.

[0063] A second registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, and the second registration relationship is used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0064] In any of the embodiments of the present application, the first coordinate system is an image coordinate system of a first CT image, the first CT image is an image collected by a CT collection device at a first time, the second coordinate system is a coordinate system established based on an optical tracking device, the n reference objects are n optical markers attached to a body surface of the first object, and the n second coordinates are determined by the optical tracking device at the first time.

[0065] The processing unit is further configured to convert the n first coordinates into n first converted coordinates in the second coordinate system based on the second registration relationship.

[0066] The processing unit is further configured to obtain a first error of the second registration relationship based on a difference between the n first converted coordinates and the n second coordinates, the difference being positively correlated with the first error.

[0067] The acquisition unit is further configured to obtain a second error of a reference registration relationship, the reference registration relationship being used to convert coordinates in an image coordinate system of a second CT image into coordinates in the second coordinate system, the second CT image being collected by the CT collection device at a second time, the reference registration relationship being obtained based on positions of the n optical markers in the image coordinate system of the second CT image and n sixth coordinates of the n optical markers in the second coordinate system, the n sixth coordinates being determined by the optical tracking device at the second time.

[0068] The processing unit is further configured to determine a difference between a breathing state of the first object at the first time and a breathing state of the first object at the second time based on a difference between the first error and the second error.

[0069] In any of the embodiments of the present application, the acquisition unit is further configured to obtain a seventh coordinate of a first optical marker in the n optical markers, the seventh coordinate being determined by the optical tracking device at a third time, a time difference between the first time and the third time being less than a sixth threshold value, the sixth threshold value being based on a duration of a breathing cycle of the first object.

[0070] The processing unit is further configured to determine an eighth coordinate from the n second coordinates, the eighth coordinate being a coordinate of the first optical marker in the second coordinate system.

[0071] The processing unit is further configured to determine an absolute value of a difference between the seventh coordinate and the eighth coordinate, to obtain a fifth absolute value.

[0072] The processing unit is further configured to obtain, based on the constraint condition, the m sets of matching coordinate pairs and the set of matching coordinate pairs, the second registration relationship.

[0073] The processing unit is further configured to obtain, based on the constraint condition, the m sets of matching coordinate pairs and the set of matching coordinate pairs, the second registration relationship.

[0074] In a third aspect, a surgical robot is provided, comprising the device for graphical registration as in the second aspect. In the third aspect, the surgical robot can perform the method for graphical registration based on the device for graphical registration, and the following effects can be achieved: improving the efficiency of determining the matching relationship between the coordinates in different coordinate systems.

[0075] In a fourth aspect, an electronic device is provided, comprising a processor and a memory, the memory being configured to store computer program code, the computer program code comprising computer instructions, and the electronic device being configured to perform the method of the first aspect and any possible implementation manner thereof when the processor executes the computer instructions.

[0076] In a fifth aspect, another electronic device is provided, comprising a processor, a sending device, an input device, an output device and a memory, the memory being configured to store computer program code, the computer program code comprising computer instructions, and the electronic device being configured to perform the method of the first aspect and any possible implementation manner thereof when the processor executes the computer instructions.

[0077] In a sixth aspect, a computer readable storage medium is provided, the computer readable storage medium storing a computer program, the computer program comprising program instructions, and the program instructions causing a processor to perform the method of the first aspect and any possible implementation manner thereof when the program instructions are executed by the processor.

[0078] In a seventh aspect, a computer program product is provided, the computer program product comprising a computer program or instructions, and the computer program or instructions causing a computer to perform the method of the first aspect and any possible implementation manner thereof when the computer program or instructions are executed on the computer.

[0079] It should be understood that the general description above and the detailed description below are only exemplary and explanatory, rather than limiting the present application.

[0080] In the embodiments of the present application, the n first coordinates are the coordinates of the n reference objects in the first coordinate system, and the n second coordinates are the coordinates of the n reference objects in the second coordinate system. After the registration device obtains the n first coordinates and the n second coordinates, the registration device determines a first coordinate set from the n first coordinates and a second coordinate set from the n second coordinates, wherein the first coordinate set includes m first coordinates from the n first coordinates, the second coordinate set includes m second coordinates from the n second coordinates, and m is an integer greater than or equal to 3 and less than or equal to n. Then, a first graph is obtained based on the coordinates in the first coordinate set, and a second graph is obtained based on the coordinates in the second coordinate set, wherein the shape of the first graph is the same as the shape of the second graph. Finally, a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set can be obtained based on the first graph and the second graph, thereby improving the efficiency of determining the matching relationship between the coordinates in different coordinate systems. BRIEF DESCRIPTION OF DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background art, the drawings needed to be used in the embodiments of the present application or the background art will be described below.

[0082] The drawings herein are incorporated into the specification and form part of the specification, which show embodiments consistent with the present application, and together with the specification, serve to explain the technical solutions of the present application.

[0083] Figure 1 A flowchart of a method for graphic registration provided in the embodiments of the present application;

[0084] Figure 2 A structural schematic diagram of a device for graphic registration provided in the embodiments of the present application;

[0085] Figure 3 A hardware structural schematic diagram of an electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0086] In order to enable persons skilled in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor fall within the scope of protection of the present application.

[0087] The terms "first", "second", and the like in the description and in the claims of the present application and above-described drawings are used for distinguishing between similar objects, not necessarily described in a particular order. Also, the terms "comprise", "comprising", and the like are intended to encompass non-exclusive inclusions. For example, processes, methods, articles, or apparatuses that comprise a list of steps or elements are not limited to the listed steps or elements, but can also comprise additional steps or elements not expressly listed, or can also comprise steps or elements inherent in such processes, methods, articles, or apparatuses.

[0088] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the embodiments described herein are combinable with each other. It is to be understood that the terms "at least one", "one or more", "at least two" or "two or more" mean one or more than one.

[0089] The execution subject of the embodiments of the present application is a device based on graphic registration (hereinafter referred to as a registration device), wherein the registration device can be any electronic device capable of executing the technical solutions disclosed in the embodiments of the method of the present application. Alternatively, the registration device can be one of the following: a computer, a server.

[0090] It should be understood that the embodiments of the method of the present application can also be implemented by a processor executing computer program code. The embodiments of the present application are described below in conjunction with the accompanying drawings in the embodiments of the present application. Please refer to Figure 1 , Figure 1 A flowchart of a method based on graphic registration provided by the embodiments of the present application.

[0091] 101、obtaining n first coordinates and n second coordinates, wherein the n first coordinates are coordinates of n reference objects in a first coordinate system, the n second coordinates are coordinates of the n reference objects in a second coordinate system, and n is an integer greater than 3.

[0092] In the embodiments of the present application, the first coordinate system and the second coordinate system are two different coordinate systems. In some schemes, the first coordinate system is an image coordinate system based on a CT image collected by a CT collection device, wherein the CT collection device is a device for collecting the CT image. The second coordinate system is a coordinate system established based on an optical tracking device, which can be used to determine the position of an optical marker in the second coordinate system.

[0093] The n references can be any n objects. Optionally, the n first coordinates and the n second coordinates are coordinates of the n objects in the first coordinate system and the second coordinate system respectively at the same time.

[0094] It should be understood that although the n first coordinates are coordinates of the n references in the first coordinate system, in step 101, for the registration device, the correspondence between the n first coordinates and the n references is unknown, for example, the n references include reference o1 and reference o2, and the n first coordinates include first coordinate z1 and first coordinate z2. The registration device cannot determine whether the first coordinate z1 is the coordinate of the reference o1 in the first coordinate system or the first coordinate z2 is the coordinate of the reference o1 in the first coordinate system based on the n first coordinates. Similarly, in step 101, for the registration device, the correspondence between the n second coordinates and the n references is also unknown.

[0095] 102. Determine a first coordinate set from the n first coordinates, wherein the first coordinate set includes m first coordinates of the n first coordinates, and m is an integer greater than or equal to 3 and less than or equal to n.

[0096] In some schemes, the registration device optionally selects m first coordinates from the n first coordinates, and obtains the first coordinate set based on the selected m first coordinates. Wherein m is an integer greater than or equal to 3 and less than or equal to n, that is, m is an integer, and the minimum value of m is 3, and the maximum value of m is n.

[0097] 103. Determine a second coordinate set from the n second coordinates, wherein the second coordinate set includes m second coordinates of the n second coordinates.

[0098] In some schemes, the registration device optionally selects m second coordinates from the n second coordinates, and obtains the second coordinate set based on the selected m second coordinates.

[0099] 104. Obtain a first graph based on the coordinates in the first coordinate set.

[0100] One coordinate in the first coordinate set represents one point in the first coordinate system, and m coordinates in the first coordinate set represent m different points in the first coordinate system. The registration device obtains the first graph based on the coordinates in the first coordinate set, that is, obtains the first graph based on the m points represented by the m coordinates in the first coordinate set.

[0101] In some schemes, m is 3, and the m coordinates in the first coordinate set represent 3 different points. The registration device can obtain a triangle based on the 3 different points, wherein the 3 different points are respectively three vertices of the triangle, and at this time the first graph is a triangle.

[0102] In some other solutions, m is 3, the m coordinates in the first coordinate set represent 3 different points, and the 3 different points are not collinear. The registration device can obtain a circle based on the 3 different points, and the first figure is a circle.

[0103] In yet some other solutions, m is 4, the m coordinates in the first coordinate set represent 4 different points. The registration device can obtain a rectangle based on the 4 different points, where the 4 different points are respectively four vertices of the rectangle, and the first figure is a rectangle.

[0104] 105. Obtain a second figure based on the coordinates in the second coordinate set, where the shape of the first figure is the same as the shape of the second figure.

[0105] The registration device obtains the second figure in the same way as obtaining the first figure, so the shape of the first figure is the same as the shape of the second figure. For example, when the first figure is a triangle, the second figure is also a triangle. When the first figure is a rectangle, the second figure is also a rectangle.

[0106] Optionally, the first figure and the second figure are both closed figures. Since at least 3 points are needed to determine a closed figure, m is greater than or equal to 3, that is, the coordinates in the first coordinate set and the coordinates in the second coordinate set are both at least 3.

[0107] 106. Obtain a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first figure and the second figure.

[0108] In the embodiments of the present application, the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set includes a matching coordinate pair in the first coordinate set and the second coordinate set, or the matching relationship includes that there is no matching coordinate pair in the first coordinate set and the second coordinate set. The matching coordinate pair includes one coordinate in the first coordinate set and one coordinate in the second coordinate set, and the two coordinates in the matching coordinate pair correspond to the same reference object in the n reference objects. For example, the first coordinate z1 in the first coordinate set is the coordinate of the reference object o1 in the first coordinate system, and the second coordinate z2 in the second coordinate set is the coordinate of the reference object o2 in the second coordinate system, so the first coordinate z1 and the second coordinate z2 are a matching coordinate pair.

[0109] If a pattern formed by the m reference objects corresponding to the m coordinates in the first coordinate set is called a first reference pattern, then the first pattern has a high similarity with the first reference pattern. If a pattern formed by the m reference objects corresponding to the m coordinates in the second coordinate set is called a second reference pattern, then the second pattern has a high similarity with the second reference pattern. Therefore, if the first pattern and the second pattern are similar, it means that the first reference pattern and the second reference pattern are similar, i.e., the probability that the m reference objects forming the first reference pattern and the m reference objects forming the second reference pattern are the same m reference objects is high, and the probability that the position of any one of the m reference objects in the first reference pattern is the same as the position of the any one of the m reference objects in the second reference pattern is high. Conversely, if the first pattern and the second pattern are not similar, it means that the first reference pattern and the second reference pattern are not similar, i.e., the probability that the m reference objects forming the first reference pattern and the m reference objects forming the second reference pattern are different m reference objects is high. Therefore, the registration device can obtain the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first pattern and the second pattern.

[0110] In Figure 1 In the method of registration based on patterns shown in the figure, the n first coordinates are the coordinates of the n reference objects in a first coordinate system, and the n second coordinates are the coordinates of the n reference objects in a second coordinate system. After obtaining the n first coordinates and the n second coordinates, the registration device determines a first coordinate set from the n first coordinates and a second coordinate set from the n second coordinates, wherein the first coordinate set includes m first coordinates of the n first coordinates, the second coordinate set includes m second coordinates of the n second coordinates, and m is an integer greater than or equal to 3 and less than or equal to n. Then, a first pattern is obtained based on the coordinates in the first coordinate set, and a second pattern is obtained based on the coordinates in the second coordinate set, wherein the shape of the first pattern is the same as the shape of the second pattern. Finally, the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set can be obtained based on the first pattern and the second pattern, thereby improving the efficiency of determining the matching relationship between the coordinates in different coordinate systems.

[0111] As an optional implementation, step 106 includes the following steps:

[0112] 1061, determining the absolute value of the difference between the perimeters of the first pattern and the second pattern to obtain a first absolute value.

[0113] 1062, in the case where the first condition is met, determining that the matching relationship includes m groups of matching coordinate pairs.

[0114] The first condition comprises that the first absolute value is less than or equal to a first threshold. The first condition being met indicates that the perimeters of the first figure and the second figure are relatively close, and thus the probability of the first figure and the second figure being similar is relatively high. In this case, m sets of matching coordinate pairs in the first coordinate set and the second coordinate set can be obtained based on the positions of the coordinates in the first coordinate set in the first figure and the positions of the coordinates in the second coordinate set in the second figure. For example, the first figure and the second figure are both triangles, the first coordinate set comprises a first coordinate z1, a first coordinate z2 and a first coordinate z3, and the second coordinate set comprises a second coordinate z4, a second coordinate z5 and a second coordinate z6. The first coordinate z1 corresponds to the largest internal angle in the first figure, the first coordinate z2 corresponds to the second largest internal angle in the first figure, and the first coordinate z3 corresponds to the smallest internal angle in the first figure. The second coordinate z4 corresponds to the largest internal angle in the second figure, the second coordinate z5 corresponds to the second largest internal angle in the second figure, and the second coordinate z6 corresponds to the smallest internal angle in the second figure. In this case, the first coordinate z1 and the second coordinate z4 can be determined as a set of matching coordinate pairs, the first coordinate z2 and the second coordinate z5 can be determined as a set of matching coordinate pairs, and the first coordinate z3 and the second coordinate z6 can be determined as a set of matching coordinate pairs.

[0115] 1063、In a case where the first condition is not met, it is determined that the matching relationship is that there is no matching coordinate pair in the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0116] The first condition not being met indicates that the perimeters of the first figure and the second figure are relatively different, and thus the probability of the first figure and the second figure being similar is relatively low. In this case, it can be determined that there is no matching coordinate pair in the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0117] In this embodiment, the registration device can determine whether the first figure and the second figure are similar based on the difference between the perimeter of the first figure and the perimeter of the second figure, and thus the matching relationship of the coordinates in the first coordinate set and the coordinates in the second coordinate set can be determined, and the efficiency of determining the matching relationship can be improved.

[0118] As an optional embodiment, m is 3, and the shape of the first figure and the shape of the second figure are both triangles. The first condition further comprises at least two of the following: the second absolute value is less than or equal to a second threshold, the third absolute value is less than or equal to a third threshold, and the fourth absolute value is less than or equal to a fourth threshold. The second absolute value is the absolute value of the difference between the largest internal angle in the first figure and the largest internal angle in the second figure, the third absolute value is the absolute value of the difference between the second largest internal angle in the first figure and the second largest internal angle in the second figure, and the fourth absolute value is the absolute value of the difference between the smallest internal angle in the first figure and the smallest internal angle in the second figure.

[0119] In the embodiment, the registration device can determine whether the first pattern and the second pattern are similar based on the difference between the perimeters of the first pattern and the second pattern and the difference between the interior angles in the first pattern and the interior angles in the second pattern, thereby improving the accuracy of the determination. Thus, the accuracy of the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set can be improved.

[0120] As an optional embodiment, after obtaining the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set, the registration device further performs the following step: in a case where the matching relationship includes m groups of matching coordinate pairs, obtaining a first registration relationship between the first coordinate system and the second coordinate system based on the m groups of matching coordinate pairs, wherein the first registration relationship is used to convert coordinates in the first coordinate system into coordinates in the second coordinate system.

[0121] In some schemes, the registration device obtains the first registration relationship by processing the m groups of matching coordinate pairs based on an iterative closest point (ICP) algorithm. Specifically, in the process of processing the m groups of matching coordinate pairs based on the ICP algorithm, multiple iterations are performed, and each iteration can obtain a first candidate registration relationship between the first coordinate system and the second coordinate system. Optionally, in the process of processing the m groups of matching coordinate pairs based on the ICP algorithm, the processing is stopped if a second condition is met, wherein the second condition includes at least one of the following: a rate of change of an error of a first candidate registration relationship obtained by two adjacent iterations is less than or equal to an error rate threshold, the error of the first candidate registration relationship is less than or equal to an error threshold, and a number of iterations based on the ICP algorithm is less than or equal to an iteration number threshold. The error of the first candidate registration relationship is a root mean square error (RMS) of n second coordinates and n third converted coordinates, wherein the third converted coordinates are obtained by converting the first coordinates into coordinates in the second coordinate system based on the first candidate registration relationship. After stopping the processing of the m groups of matching coordinate pairs based on the ICP algorithm, the first registration relationship can be obtained based on the multiple first candidate registration relationships.

[0122] Optionally, the registration device determines a first candidate registration relationship with the largest number of registration points in the multiple first candidate registration relationships as the first registration relationship, wherein a registration point of the first candidate registration relationship refers to a difference between the first coordinate and the third converted coordinate based on the first candidate registration relationship being less than or equal to a registration threshold.

[0123] Optionally, n is greater than 3, and n=6 is exemplary. The registration device determines at least one second candidate registration relationship with the number of registration points greater than 3 from the plurality of first candidate registration relationships. In the case that the number of second candidate registration relationships is 1, the second candidate registration relationship is determined as the first registration relationship. In the case that the number of second candidate registration relationships is greater than 1, and the number of second candidate registration relationships with the minimum error in the at least two second candidate registration relationships is 1, the second candidate registration relationship with the minimum error in the at least two second candidate registration relationships is determined as the first registration relationship. In the case that the number of second candidate registration relationships is greater than 1, and the number of second candidate registration relationships with the minimum error in the at least two second candidate registration relationships is greater than 1, the second candidate registration relationship with the minimum iteration number in the second candidate registration relationship with the minimum error is determined as the first registration relationship, wherein the iteration number of the second candidate registration relationship refers to the number of iterations completed when the second candidate registration relationship is obtained.

[0124] In some other schemes, in the case that m=3, the first graph and the second graph are both triangles, the registration device processes the m sets of matching coordinate pairs based on a singular value decomposition (SVD) algorithm to obtain the first registration relationship.

[0125] Optionally, the registration device calls multiple threads to process the m sets of matching coordinate pairs in parallel to obtain the first registration relationship, so as to improve the speed of obtaining the first registration relationship.

[0126] As an optional implementation, after the first registration relationship is obtained, the registration device further performs the following steps: converting a third coordinate based on the first registration relationship to obtain a fourth coordinate, wherein the third coordinate is one coordinate in the n first coordinates except the coordinates in the first coordinate set. Determining n-m second coordinates in the n second coordinates except the coordinates in the second coordinate set. In the case that a fifth coordinate exists in the n-m second coordinates, obtaining a set of matching coordinate pairs, wherein the distance between the fifth coordinate and the third coordinate is less than or equal to a fifth threshold, and the set of matching coordinate pairs includes the third coordinate and the fifth coordinate. Obtaining a second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, wherein the second registration relationship is used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0127] In the embodiment, after obtaining the first registration relationship, the registration device further determines whether there is a matching coordinate pair other than the m matching coordinate pairs in the first coordinate set and the second coordinate set by using the third coordinate on the first registration relationship. Specifically, first, the third coordinate is converted into a fourth coordinate in the second coordinate system based on the first registration relationship. If there is a coordinate in the n-m second coordinates that matches the third coordinate, there should be a coordinate in the n-m second coordinates that is closer to the fourth coordinate, where the coordinate closer to the fourth coordinate is the fifth coordinate. Therefore, the registration device determines that the third coordinate and the fifth coordinate are a matching coordinate pair when the fifth coordinate exists in the n-m second coordinates. Then, the registration relationship between the first coordinate system and the second coordinate system is recalculated based on the m matching coordinate pairs and the newly obtained matching coordinate pair to obtain a second registration relationship.

[0128] Alternatively, if the fifth coordinate does not exist in the n-m second coordinates, it indicates that the error of the first registration relationship is large, and the matching coordinate pair should be determined again from the n first coordinates and the n second coordinates. For example, m=3, and both the first figure and the second figure are triangles. If at least two similar triangles can be obtained based on the n reference objects, the 3 reference objects corresponding to the first figure and the 3 reference objects corresponding to the second figure can be different, but the similarity of the first figure and the second figure is high. At this time, it is easy to cause a large error in the first registration relationship. Therefore, the registration device determines a third coordinate set from the n first coordinates and a fourth coordinate set from the n second coordinates when the fifth coordinate does not exist in the n-m second coordinates. A third figure is obtained based on the third coordinate set, and a fourth figure is obtained based on the fourth coordinate set. A matching relationship between the coordinates in the third coordinate set and the coordinates in the fourth coordinate set is obtained based on the third figure and the fourth figure. In the case where the matching relationship between the coordinates in the third coordinate set and the coordinates in the fourth coordinate set includes m matching coordinate pairs (to distinguish the m matching coordinate pairs in the first coordinate set and the second coordinate set, the m matching coordinate pairs in the first coordinate set and the second coordinate set are referred to as new m matching coordinate pairs below), a third registration relationship between the first coordinate system and the second coordinate system is obtained based on the new m matching coordinate pairs. In this way, the accuracy of the registration relationship between the first coordinate system and the second coordinate system can be improved.

[0129] As an optional embodiment, the first coordinate system is an image coordinate system of a first CT image, where the first CT image is an image collected by a CT collection device at a first time, the second coordinate system is a coordinate system established based on an optical tracking device, and the n reference objects are n optical markers attached to the surface of the first object. The n second coordinates are determined by the optical tracking device at the first time.

[0130] In the implementation, the registration device further performs the following steps: converting the n first coordinates into n first converted coordinates in the second coordinate system based on the second registration relationship. The first error of the second registration relationship is obtained based on a difference between the n first converted coordinates and the n second coordinates, wherein the difference is positively correlated with the first error. The second error of the reference registration relationship is obtained, wherein the reference registration relationship is used to convert coordinates in an image coordinate system of a second CT image into coordinates in the second coordinate system, the second CT image is collected by the CT collection device at a second time, the reference registration relationship is obtained based on positions of the n optical markers in the image coordinate system of the second CT image and n sixth coordinates of the n optical markers in the second coordinate system, and the n sixth coordinates are determined by the optical tracking device at the second time. The difference between the first error and the second error is used to determine the difference between the respiratory state of the first subject at the first time and the respiratory state of the first subject at the second time.

[0131] In the implementation, the respiration of the first subject causes the body surface of the first subject to move, and further causes the positions of the optical markers in the body surface to change at different times, wherein the respiratory state at a time refers to a phase corresponding to the time in a respiratory cycle of the first subject. Because the registration relationship between the image coordinate system of the CT image and the second coordinate system is obtained based on the positions of the optical markers, the errors of the registration relationships obtained at different times corresponding to different respiratory states are different. Conversely, the difference between the errors of the two registration relationships can be used to determine the difference between the respiratory states of the two registration relationships at the times.

[0132] In a possible implementation scenario, in the medical field, it is usually necessary to plan a target path from a skin region of the first subject to a tissue in the first subject. The CT image collected by the CT collection device includes information of the skin region of the first subject and information of the tissue (such as a liver) in the first subject, and therefore the target path can be planned based on the CT image of the first subject. Specifically, the target path can be planned based on the second CT image. After the target path is obtained, the target object can be controlled to move along the target path from the skin region of the first subject to the tissue in the first subject based on the mechanical arm, and the target object is exemplarily a needle.

[0133] Because the coordinate system established based on the optical tracking device is the same as the world coordinate system, that is, the second coordinate system is the same as the world coordinate system, converting the target path in the CT image into the second coordinate system is beneficial to controlling the target object to move along the target path based on the mechanical arm and is convenient for a doctor to observe the relative positional relationship between the target path and the first subject. Therefore, the reference registration relationship needs to be obtained based on the positions of the n optical markers in the image coordinate system of the second CT image and the n sixth coordinates.

[0134] Considering that the respiration of the first object causes the body surface of the first object and the tissue inside the first object to generate movement, which in turn causes the respiration state at different time instants to be different, and thus causes the path from the skin region of the body surface to the tissue inside the body to be different at different time instants, when the second CT image is acquired at a second time instant and the n sixth coordinates are determined at the second time instant, the time instant at which the respiration state is close to the respiration state at the second time instant should be determined as the time instant at which the target object is controlled by the robot arm to move from the skin region of the first object to the tissue inside the first object along the target path (hereinafter referred to as the moving time instant). In this way, the success rate of moving from the skin region of the first object to the tissue inside the first object along the target path can be improved.

[0135] According to the embodiment, the difference between the respiration states at the time instants at which the two registration relationships are determined can be determined, and thus the moving time instant can be determined based on the embodiment. Optionally, the registration relationship between the image coordinate system of the CT image acquired at the moving time instant and the second coordinate system is referred to as a moving registration relationship. The difference between the error of the moving registration relationship and the error of the reference registration relationship is less than or equal to the eighth threshold value.

[0136] In some schemes, after the target path is obtained, the CT acquisition device periodically acquires CT images of the first object, and the registration device determines the coordinates of the n optical markers in the image coordinate system of each CT image, and the optical tracking device periodically determines the coordinates of the n optical markers in the second coordinate system. Then, the moving time instant can be determined based on all the CT images and the coordinates determined by the optical tracking device.

[0137] Optionally, because the respiration of the first object is periodic, in order to reduce the storage overhead of the CT images acquired by the CT acquisition device and the coordinates determined by the optical tracking device, the registration device stores the CT images acquired at a time instant that is not more than a preset time length from the current time instant and the coordinates determined at a time instant that is not more than the preset time length from the current time instant. The preset time length is the duration of the respiration cycle of the first object. For example, the preset time length is 5 seconds.

[0138] As an optional implementation, before performing the step of "obtaining the second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matched coordinate pairs and the set of matched coordinate pairs", the registration device further performs the following steps: obtaining a seventh coordinate of a first optical marker in the n optical markers, wherein the seventh coordinate is determined by the optical tracking device at a third time, a time difference between the first time and the third time is less than a sixth threshold value, and the sixth threshold value is based on a duration of a breathing cycle of the first object; determining an eighth coordinate from the n second coordinates, wherein the eighth coordinate is a coordinate of the first optical marker in the second coordinate system; determining an absolute value of a difference between the seventh coordinate and the eighth coordinate to obtain a fifth absolute value; and obtaining a constraint condition for the registration relationship between the first coordinate system and the second coordinate system based on the fifth absolute value, wherein the constraint condition comprises that an absolute value of a difference between a second converted coordinate and the eighth coordinate is less than or equal to a seventh threshold value, the second converted coordinate is obtained by converting the seventh coordinate into a coordinate in the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system, and the seventh threshold value is positively correlated with the fifth absolute value. After obtaining the constraint condition, the following steps are performed in the process of performing the step of "obtaining the second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matched coordinate pairs and the set of matched coordinate pairs": obtaining the second registration relationship based on the constraint condition, the m sets of matched coordinate pairs and the set of matched coordinate pairs.

[0139] In this implementation, the smaller the fifth absolute value is, the smaller the difference between the positions of the first optical marker at different times is, that is, the smaller the influence of the breathing of the first object on the first optical marker is, and thus the higher the stability of the coordinates of the first optical marker is. In order to improve the accuracy of the registration relationship between the first coordinate system and the second coordinate system, the registration relationship can be constrained based on the seventh coordinate of the first optical marker in the first coordinate system and the eighth coordinate of the first optical marker in the second coordinate system. Specifically, the coordinate obtained by converting the seventh coordinate into a coordinate in the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system is a second converted coordinate, and thus the absolute value of the difference between the second converted coordinate and the eighth coordinate represents the conversion error of the registration relationship on the coordinates of the first optical marker. The higher the stability of the coordinates of the first optical marker is, the smaller the conversion error should be, which can effectively improve the accuracy of the registration relationship. Therefore, the constraint condition comprises that the absolute value of the difference between the second converted coordinate and the eighth coordinate is less than or equal to the seventh threshold value, and the seventh threshold value is positively correlated with the fifth absolute value.

[0140] After obtaining the constraint condition, the second registration relationship is obtained based on the constraint condition, the m sets of matched coordinate pairs and the set of matched coordinate pairs, which can improve the accuracy of the second registration relationship. Specifically, the registration device obtains the second registration relationship satisfying the constraint condition based on the m sets of matched coordinate pairs and the set of matched coordinate pairs.

[0141] In some solutions, the seventh threshold value is further related to at least one of a confidence of the matching coordinate pair corresponding to the first optical marker, and a probability of the first optical marker being blocked, wherein the seventh threshold value is negatively related to the confidence of the matching coordinate pair corresponding to the first optical marker, and the seventh threshold value is positively related to the probability of the first optical marker being blocked. Specifically, the matching coordinate pair corresponding to the first optical marker is the matching coordinate pair including the seventh coordinate and the eighth coordinate in the m matching coordinate pairs. The confidence of the matching coordinate pair corresponding to the first optical marker is negatively related to the first absolute value. The higher the confidence of the matching coordinate pair corresponding to the first optical marker, the smaller the conversion error should be, which can effectively improve the accuracy of the registration relationship. The first optical marker being blocked means that there is an occlusion between the optical tracking device and the first optical marker, and the existence of the occlusion will cause the error of the eighth coordinate determined by the optical tracking device to be large.

[0142] Optionally, represents the seventh threshold value, represents the stability of the coordinate of the first optical marker, represents the confidence of the matching coordinate pair corresponding to the first optical marker, represents the probability of the first optical marker being blocked, and then , , satisfies the following formula:

[0143] … formula (1)

[0144] Optionally, is calculated by the following formula:

[0145] … formula (2)

[0146] wherein, represents the fifth absolute value.

[0147] Those skilled in the art can understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined by its function and possible internal logic.

[0148] If the technical solution of the application involves personal information, the product applying the technical solution of the application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solution of the application involves sensitive personal information, the product applying the technical solution of the application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent mark is set to inform that the personal information collection range has been entered, and the personal information will be collected. If the individual voluntarily enters the collection range, it is regarded as consent to collect personal information; or on the device for processing personal information, the personal information processing rules are informed by using obvious marks / information, and the personal authorization is obtained by means of pop-up information or asking the individual to upload his / her personal information, etc. Wherein, the personal information processing may include personal information processor, personal information processing purpose, processing method and personal information type, etc.

[0149] The above describes the method of the embodiment of the application in detail, and the device of the embodiment of the application is provided below.

[0150] Please refer to Figure 2 , Figure 2 A structure schematic diagram of a device based on graphic registration provided by the embodiment of the application. The device based on graphic registration 1 comprises an acquisition unit 11 and a processing unit 12, specifically:

[0151] The acquisition unit 11 is configured to acquire n first coordinates and n second coordinates, the n first coordinates being coordinates of n reference objects in a first coordinate system, the n second coordinates being coordinates of the n reference objects in a second coordinate system, and the n being an integer greater than 3;

[0152] The processing unit 12 is configured to determine a first coordinate set from the n first coordinates, the first coordinate set comprising m first coordinates in the n first coordinates, and the m being an integer greater than or equal to 3 and less than or equal to the n;

[0153] The processing unit 12 is further configured to determine a second coordinate set from the n second coordinates, the second coordinate set comprising m second coordinates in the n second coordinates;

[0154] The processing unit 12 is further configured to obtain a first graph based on the coordinates in the first coordinate set;

[0155] The processing unit 12 is further configured to obtain a second graph based on the coordinates in the second coordinate set, the shape of the first graph being the same as the shape of the second graph;

[0156] The processing unit 12 is further configured to obtain a matching relationship between a coordinate in the first coordinate set and a coordinate in the second coordinate set based on the first graph and the second graph.

[0157] In any of the embodiments of the present application, the processing unit 12 is further configured to:

[0158] determine a first absolute value of a difference between a perimeter of the first graph and a perimeter of the second graph;

[0159] In a case where a first condition is met, the matching relationship includes m groups of matching coordinate pairs.

[0160] The first condition includes that the first absolute value is less than or equal to a first threshold value. The matching coordinate pair includes one coordinate in the first coordinate set and one coordinate in the second coordinate set, and the two coordinates in the matching coordinate pair correspond to a same one of the n reference objects. The m groups of matching coordinate pairs are obtained based on positions of the coordinates in the first coordinate set in the first graph and positions of the coordinates in the second coordinate set in the second graph.

[0161] In a case where the first condition is not met, the matching relationship is that there is no matching coordinate pair between the coordinates in the first coordinate set and the coordinates in the second coordinate set.

[0162] In any of the embodiments of the present application, the m is 3, and shapes of the first graph and the second graph are both triangles.

[0163] The first condition further includes at least two of the following: a second absolute value is less than or equal to a second threshold value, a third absolute value is less than or equal to a third threshold value, and a fourth absolute value is less than or equal to a fourth threshold value.

[0164] The second absolute value is an absolute value of a difference between a largest interior angle of the first graph and a largest interior angle of the second graph. The third absolute value is an absolute value of a difference between a second largest interior angle of the first graph and a second largest interior angle of the second graph. The fourth absolute value is an absolute value of a difference between a smallest interior angle of the first graph and a smallest interior angle of the second graph.

[0165] In any of the embodiments of the present application, the processing unit 12 is further configured to:

[0166] In a case where the matching relationship includes the m groups of matching coordinate pairs, the processing unit 12 is further configured to obtain a first registration relationship between the first coordinate system and the second coordinate system based on the m groups of matching coordinate pairs, the first registration relationship being used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0167] In combination with any of the embodiments of the present application, the m is less than the n, and the processing unit 12 is further configured to:

[0168] convert a third coordinate into a fourth coordinate based on the first registration relationship, the third coordinate being one of the n first coordinates other than the coordinates in the first coordinate set;

[0169] determine n-m second coordinates of the n second coordinates other than the coordinates in the second coordinate set;

[0170] in a case where a fifth coordinate exists in the n-m second coordinates, the fifth coordinate being less than or equal to a fifth threshold value from the third coordinate, obtain a set of matching coordinate pairs, the set of matching coordinate pairs including the third coordinate and the fifth coordinate;

[0171] obtain a second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, the second registration relationship being used to convert a coordinate in the first coordinate system into a coordinate in the second coordinate system.

[0172] In combination with any of the embodiments of the present application, the first coordinate system is an image coordinate system of a first CT image, the first CT image being an image collected by a CT collection device at a first time, the second coordinate system is a coordinate system established based on an optical tracking device, the n reference objects are n optical markers attached to a body surface of a first object, and the n second coordinates are determined by the optical tracking device at the first time.

[0173] The processing unit 12 is further configured to convert the n first coordinates into n first converted coordinates in the second coordinate system based on the second registration relationship.

[0174] The processing unit 12 is further configured to obtain a first error of the second registration relationship based on a difference between the n first converted coordinates and the n second coordinates, the difference being positively correlated with the first error.

[0175] The acquisition unit 11 is further configured to obtain a second error of a reference registration relationship, the reference registration relationship being used to convert a coordinate in an image coordinate system of a second CT image into a coordinate in the second coordinate system, the second CT image being collected by the CT collection device at a second time, the reference registration relationship being obtained based on positions of the n optical markers in the image coordinate system of the second CT image and n sixth coordinates of the n optical markers in the second coordinate system, the n sixth coordinates being determined by the optical tracking device at the second time.

[0176] The processing unit 12 is further configured to determine a difference between the respiratory state of the first object at the first time and the respiratory state of the first object at the second time based on a difference between the first error and the second error.

[0177] In combination with any of the embodiments of the present application, the acquisition unit 11 is further configured to acquire a seventh coordinate of a first optical marker in the n optical markers, the seventh coordinate being determined by the optical tracking device at a third time, a time difference between the first time and the third time being less than a sixth threshold value, the sixth threshold value being based on a duration of a respiratory cycle of the first object.

[0178] The processing unit 12 is further configured to determine an eighth coordinate from the n second coordinates, the eighth coordinate being a coordinate of the first optical marker in the second coordinate system.

[0179] The processing unit 12 is further configured to determine an absolute value of a difference between the seventh coordinate and the eighth coordinate, obtaining a fifth absolute value.

[0180] The processing unit 12 is further configured to obtain, based on the fifth absolute value, a constraint condition that the constraint condition is a registration relationship between the first coordinate system and the second coordinate system, the constraint condition including that an absolute value of a difference between a second conversion coordinate and the eighth coordinate is less than or equal to a seventh threshold value, the second conversion coordinate being obtained by converting the seventh coordinate into a coordinate in the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system, the seventh threshold value being positively correlated with the fifth absolute value.

[0181] The processing unit 12 is further configured to obtain the second registration relationship based on the constraint condition, the m sets of matching coordinate pairs, and the one set of matching coordinate pairs.

[0182] In the embodiments of the present application, the n first coordinates are coordinates of n reference objects in a first coordinate system, and the n second coordinates are coordinates of the n reference objects in a second coordinate system. After the registration device acquires the n first coordinates and the n second coordinates, the registration device determines a first coordinate set from the n first coordinates and a second coordinate set from the n second coordinates, wherein the first coordinate set includes m first coordinates in the n first coordinates, the second coordinate set includes m second coordinates in the n second coordinates, and m is an integer greater than or equal to 3 and less than or equal to n. Then, a first figure is obtained based on the coordinates in the first coordinate set, and a second figure is obtained based on the coordinates in the second coordinate set, wherein the shape of the first figure is the same as the shape of the second figure. Finally, a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set can be obtained based on the first figure and the second figure, thereby improving the efficiency of determining the matching relationship between the coordinates in different coordinate systems.

[0183] In some embodiments, the apparatus provided by the embodiments of the present application has functions or includes modules for performing the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, details are not described here.

[0184] Figure 3 A hardware structure schematic diagram of an electronic device provided by the embodiments of the present application is shown. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 further includes an input device 23 and an output device 24. The processor 21, the memory 22, the input device 23 and the output device 24 are coupled through a connector, which includes various interfaces, transmission lines or buses, etc., and the embodiments of the present application do not make any limitation. It should be understood that in various embodiments of the present application, coupling means mutual contact in a specific way, including direct connection or indirect connection through other devices, for example, various interfaces, transmission lines, buses, etc.

[0185] The processor 21 can be one or more graphics processing units (GPUs). In the case of the processor 21 being a GPU, the GPU can be a single-core GPU or a multi-core GPU. Optionally, the processor 21 can be a processor group composed of multiple GPUs, and the multiple processors are coupled with each other through one or more buses. Optionally, the processor can also be other types of processors, etc., and the embodiments of the present application do not make any limitation.

[0186] The memory 22 can be used to store computer program instructions and various computer program codes for executing the schemes of the present application. Optionally, the memory includes but is not limited to a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0187] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The input device 23 and the output device 24 can be independent devices, or can be an integral device.

[0188] It can be understood that, in the embodiments of the present application, the memory 22 can be used to store not only related instructions but also related data, and the embodiments of the present application do not limit the data stored in the memory.

[0189] It can be understood that, Figure 3 Only a simplified design of an electronic device is shown. In actual applications, the electronic device can also include other necessary elements, including but not limited to any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of the present application are within the protection scope of the present application.

[0190] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0191] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here. Those skilled in the art can also clearly understand that each embodiment of the present application describes each with emphasis, and for the convenience and brevity of description, the same or similar parts may not be described in detail in different embodiments. Therefore, the parts not described or not described in detail in a certain embodiment can be referred to the description of other embodiments.

[0192] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can be in another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0193] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0194] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0195] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital versatile disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0196] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium includes read-only memory (ROM) or random access memory (RAM), magnetic disk or optical disk, and various media that can store program codes.

Claims

1. A method based on image registration, characterized in that, The method includes: Obtain n first coordinates and n second coordinates. The n first coordinates are the coordinates of n reference objects in a first coordinate system, and the n second coordinates are the coordinates of the n reference objects in a second coordinate system. The n is an integer greater than 3. The first coordinate system is the image coordinate system of the first CT image, which is the image acquired by the CT acquisition device at the first moment. The second coordinate system is a coordinate system established based on the optical tracking device. The n reference objects are n optical markers attached to the surface of the first object. The n second coordinates are determined by the optical tracking device at the first moment. A first coordinate set is determined from the n first coordinates, the first coordinate set including m first coordinates from the n first coordinates, where m is an integer greater than or equal to 3 and less than or equal to n; A second coordinate set is determined from the n second coordinates, wherein the second coordinate set includes m second coordinates from the n second coordinates; The first graphic is obtained based on the coordinates in the first coordinate set; The second graphic is obtained based on the coordinates in the second coordinate set, and the shape of the first graphic is the same as the shape of the second graphic. Based on the first and second graphics, the matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set is obtained; When the matching relationship includes m sets of matching coordinate pairs, a first registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matching coordinate pairs. The first registration relationship is used to convert the coordinates in the first coordinate system into the coordinates in the second coordinate system. Based on the first registration relationship, the third coordinate is transformed to obtain the fourth coordinate, wherein the third coordinate is one of the n first coordinates other than the coordinates in the first coordinate set; Determine nm of the n second coordinates, excluding the coordinates in the second coordinate set; If a fifth coordinate exists among the nm second coordinates, a set of matching coordinate pairs is obtained, wherein the distance between the fifth coordinate and the fourth coordinate is less than or equal to a fifth threshold, and the set of matching coordinate pairs includes the third coordinate and the fifth coordinate; Based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, a second registration relationship between the first coordinate system and the second coordinate system is obtained. The second registration relationship is used to convert the coordinates in the first coordinate system into the coordinates in the second coordinate system. Based on the second registration relationship, the n first coordinates are converted into n first transformed coordinates in the second coordinate system; Based on the difference between the n first transformation coordinates and the n second coordinates, a first error of the second registration relationship is obtained, and the difference is positively correlated with the first error; The second error of the reference registration relationship is obtained. The reference registration relationship is used to convert the coordinates in the image coordinate system of the second CT image into coordinates in the second coordinate system. The second CT image is acquired by the CT acquisition device at the second time. The reference registration relationship is based on the positions of the n optical markers in the image coordinate system of the second CT image and the n sixth coordinates of the n optical markers in the second coordinate system. The n sixth coordinates are determined by the optical tracking device at the second time. Based on the difference between the first error and the second error, the difference between the breathing state of the first object at the first time and the breathing state of the first object at the second time is determined.

2. The method according to claim 1, characterized in that, The step of obtaining the matching relationship between coordinates in the first coordinate set and coordinates in the second coordinate set based on the first and second graphics includes: Determine the absolute value of the difference between the perimeter of the first figure and the perimeter of the second figure to obtain the first absolute value; If the first condition is met, the matching relationship is determined to include m sets of matching coordinate pairs; Wherein, the first condition includes the first absolute value being less than or equal to a first threshold; the matching coordinate pair includes a coordinate from the first coordinate set and a coordinate from the second coordinate set, and the two coordinates in the matching coordinate pair correspond to the same reference object among the n reference objects; the m sets of matching coordinate pairs are obtained based on the position of the coordinate in the first coordinate set in the first graphic and the position of the coordinate in the second coordinate set in the second graphic; If the first condition is not met, the matching relationship is determined to be that there is no matching coordinate pair in the coordinates of the first coordinate set and the coordinates of the second coordinate set.

3. The method according to claim 2, characterized in that, The value of m is 3, and the shapes of both the first and second figures are triangles. The first condition also includes at least two of the following: the second absolute value is less than or equal to the second threshold, the third absolute value is less than or equal to the third threshold, and the fourth absolute value is less than or equal to the fourth threshold; Wherein, the second absolute value is the absolute value of the difference between the largest interior angle in the first figure and the largest interior angle in the second figure, the third absolute value is the absolute value of the difference between the second largest interior angle in the first figure and the second largest interior angle in the second figure, and the fourth absolute value is the absolute value of the difference between the smallest interior angle in the first figure and the smallest interior angle in the second figure.

4. The method according to claim 1, characterized in that, Before obtaining the second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, the method further includes: The seventh coordinate of the first optical marker among the n optical markers is obtained. The seventh coordinate is determined by the optical tracking device at a third time. The time difference between the first time and the third time is less than a sixth threshold. The sixth threshold is based on the duration of the respiratory cycle of the first object. An eighth coordinate is determined from the n second coordinates, wherein the eighth coordinate is the coordinate of the first optical marker in the second coordinate system; Determine the absolute value of the difference between the seventh coordinate and the eighth coordinate to obtain the fifth absolute value; Based on the fifth absolute value, the constraint conditions for the registration relationship between the first coordinate system and the second coordinate system are obtained. The constraint conditions include that the absolute value of the difference between the second transformed coordinate and the eighth coordinate is less than or equal to the seventh threshold. The second transformed coordinate is obtained by converting the seventh coordinate into coordinates in the second coordinate system based on the registration relationship between the first coordinate system and the second coordinate system. The seventh threshold is positively correlated with the fifth absolute value. The step of obtaining the second registration relationship between the first coordinate system and the second coordinate system based on the m sets of matching coordinate pairs and the first set of matching coordinate pairs includes: Based on the constraints, the m sets of matching coordinate pairs, and the set of matching coordinate pairs, the second registration relationship is obtained.

5. A device based on image registration, characterized in that, The graph-based registration device includes: An acquisition unit is used to acquire n first coordinates and n second coordinates. The n first coordinates are the coordinates of n reference objects in a first coordinate system, and the n second coordinates are the coordinates of the n reference objects in a second coordinate system. The n is an integer greater than 3. The first coordinate system is the image coordinate system of a first CT image, which is an image acquired by a CT acquisition device at a first moment. The second coordinate system is a coordinate system established based on an optical tracking device. The n reference objects are n optical markers attached to the surface of a first object. The n second coordinates are determined by the optical tracking device at the first moment. A processing unit is configured to determine a first coordinate set from the n first coordinates, wherein the first coordinate set includes m first coordinates from the n first coordinates, and m is an integer greater than or equal to 3 and less than or equal to n; The processing unit is further configured to determine a second coordinate set from the n second coordinates, wherein the second coordinate set includes m second coordinates from the n second coordinates; The processing unit is further configured to obtain a first graphic based on the coordinates in the first coordinate set; The processing unit is further configured to obtain a second graphic based on the coordinates in the second coordinate set, wherein the shape of the first graphic is the same as the shape of the second graphic; The processing unit is further configured to obtain a matching relationship between the coordinates in the first coordinate set and the coordinates in the second coordinate set based on the first graphic and the second graphic; When the matching relationship includes m sets of matching coordinate pairs, a first registration relationship between the first coordinate system and the second coordinate system is obtained based on the m sets of matching coordinate pairs. The first registration relationship is used to convert the coordinates in the first coordinate system into the coordinates in the second coordinate system. Based on the first registration relationship, the third coordinate is transformed to obtain the fourth coordinate, wherein the third coordinate is one of the n first coordinates other than the coordinates in the first coordinate set; Determine nm of the n second coordinates, excluding the coordinates in the second coordinate set; If a fifth coordinate exists among the nm second coordinates, a set of matching coordinate pairs is obtained, wherein the distance between the fifth coordinate and the fourth coordinate is less than or equal to a fifth threshold, and the set of matching coordinate pairs includes the third coordinate and the fifth coordinate; Based on the m sets of matching coordinate pairs and the set of matching coordinate pairs, a second registration relationship between the first coordinate system and the second coordinate system is obtained. The second registration relationship is used to convert the coordinates in the first coordinate system into the coordinates in the second coordinate system. Based on the second registration relationship, the n first coordinates are converted into n first transformed coordinates in the second coordinate system; Based on the difference between the n first transformation coordinates and the n second coordinates, a first error of the second registration relationship is obtained, and the difference is positively correlated with the first error; The second error of the reference registration relationship is obtained. The reference registration relationship is used to convert the coordinates in the image coordinate system of the second CT image into coordinates in the second coordinate system. The second CT image is acquired by the CT acquisition device at the second time. The reference registration relationship is based on the positions of the n optical markers in the image coordinate system of the second CT image and the n sixth coordinates of the n optical markers in the second coordinate system. The n sixth coordinates are determined by the optical tracking device at the second time. Based on the difference between the first error and the second error, the difference between the breathing state of the first object at the first time and the breathing state of the first object at the second time is determined.

6. A surgical robot, characterized in that, Includes the graphic registration-based apparatus as described in claim 5.

7. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1 to 4.

Images