An automatic registration method and apparatus, electronic device, and medium

Abstract

Embodiments of the present application provide an automatic registration method and device, electronic equipment and medium, which relate to the field of intelligent medical treatment. The embodiments of the present application include: obtaining an X-ray image obtained by shooting a calibration plate, and identifying a first position of each marker ball included in the calibration plate in an X-ray image coordinate system to obtain a first point set. The second position of each marker ball in the calibration plate coordinate system is obtained to obtain a second point set. Then, the probability of the corresponding relationship between each first position and each second position is determined to obtain a probability matrix, wherein the physical positions represented by the first position and the second position with the corresponding relationship are the same. Then, the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined according to the first point set, the second point set and the probability matrix. The present method has no complex requirements for the size and arrangement of the small balls, is more widely applicable, and improves the accuracy and robustness of determining the conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of smart medical treatment, in particular to an automatic registration method and device, electronic equipment and a medium. BACKGROUND

[0002] When performing surgery by using a surgical robot, the operation positioning is more accurate, the mechanical arm of the surgical robot can grasp the surgical tool more stably and powerfully, and the fatigue of the surgeon caused by long-time surgery can be avoided, thereby improving the precision, stability and safety of the surgery. In the process of performing surgery by using the surgical robot, a calibration plate needs to be placed near a lesion, and then an X-ray image of the calibration plate and the lesion is captured, the position of the lesion is positioned by using the X-ray image, and then surgery is performed at the position of the lesion by using a surgical tool. Therefore, in the process of surgical navigation, the conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system needs to be realized to accurately position the surgical tool.

[0003] The conversion relationship needs to be determined according to the mark points in the X-ray image and the arrangement order of the mark points. When the mark points are extracted, candidate points in the calibration plate need to be extracted from the X-ray image, and then the distance between each two candidate points is calculated, and the candidate points with a distance not meeting the condition are deleted. Then, three mark points constituting a preset included angle are detected from the screened candidate points, the fourth mark point is estimated by using the three mark points, and then the mark points are sorted and numbered.

[0004] Because the structure and the placement posture of the calibration plate can affect the positions of the mark points in the calibration plate in the X-ray image, the included angle between the mark points in the X-ray image is arbitrary, it is difficult to screen the mark points by using the preset included angle, and it is difficult to accurately sort and number the mark points. Therefore, the accuracy of the conversion relationship determined by using the method is low. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide an automatic registration method, device, electronic equipment and medium, so as to improve the accuracy of determining the conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system. The specific technical solutions are as follows:

[0006] In a first aspect, the present application provides an automatic registration method, which comprises the following steps:

[0007] An X-ray image obtained by capturing a calibration plate is acquired, and a first position of each mark ball included in the calibration plate in an X-ray image coordinate system is recognized, to obtain a first point set;

[0008] A second position of each mark ball in a calibration plate coordinate system is acquired, to obtain a second point set;

[0009] determine a probability that each first position in the first point set has a corresponding relationship with each second position in the second point set, to obtain a probability matrix, wherein the first position and the second position having the corresponding relationship represent a same physical position;

[0010] determine a coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix.

[0011] Optionally, the determining of the probability that each first position in the first point set has a corresponding relationship with each second position in the second point set comprises:

[0012] for each first position included in the first point set and each second position included in the second point set, convert the second position to the X-ray image coordinate system based on an initial coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system, to obtain a projection position, determine an error between the projection position and the first position, and determine the probability that the first position and the second position have the corresponding relationship according to the determined error.

[0013] Optionally, the converting of the second position to the X-ray image coordinate system based on the initial coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system, to obtain the projection position, the determining of the error between the projection position and the first position, and the determining of the probability that the first position and the second position have the corresponding relationship according to the determined error comprise:

[0014] the probability that each first position and each second position have the corresponding relationship is determined by the following formula:

[0015]

[0016]

[0017] wherein, m ij is the probability that the ith first position and the jth second position have the corresponding relationship, exp represents an exponential function with e as a base, α and β are both preset constants, R 0 is a rotation relationship of the initial coordinate conversion relationship, t 0 is a translation relationship of the initial coordinate conversion relationship, p i is the ith first position, y Ri is the jth second position.

[0018] Optionally, before determining the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix, the method further comprises:

[0019] performing row and column standardization on the probability matrix to obtain a row and column standardized probability matrix;

[0020] The determining the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix comprises:

[0021] determining the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the row and column standardized probability matrix.

[0022] Optionally, the determining the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix comprises:

[0023] constructing a target function according to the first point set, the second point set and the probability matrix; wherein the target function is used to determine an error between each projection position obtained by projecting each second position included in the second point set to the X-ray image coordinate system based on the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system and each first position;

[0024] solving the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system when the target function reaches a minimum value.

[0025] Optionally, the target function is:

[0026]

[0027] wherein E is a target function value, n x is a first position quantity included in the first point set, n y is a second position quantity included in the second point set, m ij is a probability that the i th first position and the j th second position have a corresponding relationship, p i is the i th first position, y Rj is the j th second position, R k-1 is a rotation relationship of the coordinate conversion relationship obtained in the (k-1) th iteration in the solving process, t k-1 is a translation relationship of the coordinate conversion relationship obtained in the (k-1) th iteration in the solving process, k is a current iteration number, and a and β are both preset constants.

[0028] Optionally, after the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined according to the first point set, the second point set and the probability matrix, the method further comprises:

[0029] projecting each second position in the second point set onto the X-ray image based on the coordinate conversion relationship;

[0030] displaying the projected X-ray image;

[0031] adjusting the first point set based on an adjustment instruction of a user;

[0032] determining a probability that each first position in the adjusted first point set has a corresponding relationship with each second position in the second point set, to obtain an updated probability matrix;

[0033] determining an updated coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the updated probability matrix.

[0034] Optionally, the adjusting the first point set based on the adjustment instruction of the user comprises:

[0035] receiving an increase instruction of the user, and adding a first position specified by the user to the first point set;

[0036] receiving a deletion instruction of the user, and deleting a first position specified by the user in the first point set;

[0037] receiving a modification instruction of the user, and modifying a first position specified by the user in the first point set; and / or,

[0038] receiving a sequence adjustment instruction of the user, and adjusting the sequence of each first position included in the first point set.

[0039] A second aspect of the embodiment of the application provides an automatic registration device, which comprises:

[0040] a recognition module, configured to acquire an X-ray image obtained by photographing a calibration plate, and recognize a first position of each marker ball included in the calibration plate in an X-ray image coordinate system, to obtain a first point set;

[0041] an acquisition module, configured to acquire a second position of each marker ball in a calibration plate coordinate system, to obtain a second point set;

[0042] a determination module, configured to determine a probability that each first position in the first point set recognized by the recognition module has a corresponding relationship with each second position in the second point set acquired by the acquisition module, to obtain a probability matrix, wherein the first position and the second position having the corresponding relationship represent the same physical position.

[0043] The determining module is further configured to determine a coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix.

[0044] The determining module is specifically configured to:

[0045] For each first position included in the first point set and each second position included in the second point set, the second position is converted to the X-ray image coordinate system based on an initial coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system to obtain a projection position, an error between the projection position and the first position is determined, and a probability that the first position and the second position have a corresponding relationship is obtained according to the determined error.

[0046] Optionally, the determining module is specifically configured to:

[0047] The probability that each first position and each second position have a corresponding relationship is determined by the following formula:

[0048]

[0049]

[0050] wherein, m ij is a probability that the ith first position and the jth second position have a corresponding relationship, exp represents an exponential function with e as a base, and α and β are both preset constants, R 0 is a rotation relationship of the initial coordinate conversion relationship, t 0 is a translation relationship of the initial coordinate conversion relationship, p i is the ith first position, and y Ri is the jth second position.

[0051] Optionally, the apparatus further includes:

[0052] A standardization module configured to perform row and column standardization on the probability matrix to obtain a row and column standardized probability matrix before determining the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix.

[0053] The determining module is specifically configured to:

[0054] The coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined according to the first point set, the second point set and the row and column standardized probability matrix.

[0055] Optionally, the determining module is specifically configured to:

[0056] construct a target function according to the first point set, the second point set and the probability matrix, wherein the target function is used to determine an error between each projection position and each first position, wherein each projection position is obtained by projecting each second position included in the second point set to the X-ray image coordinate system based on a coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system;

[0057] solving the target function to obtain a minimum value, the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system.

[0058] Optionally, the target function is:

[0059]

[0060] wherein E is a target function value, n x is a first position quantity included in the first point set, n y is a second position quantity included in the second point set, m ij is a probability that the i th first position and the j th second position have a corresponding relationship, p i is the i th first position, y Rj is the j th second position, R k-1 is a rotation relationship of a coordinate conversion relationship obtained in a (k-1) th iteration in a solving process, t k-1 is a translation relationship of the coordinate conversion relationship obtained in the (k-1) th iteration in the solving process, k is a current iteration number, and α and β are both preset constants.

[0061] Optionally, the device further comprises:

[0062] a projection module, configured to project each second position in the second point set to the X-ray image based on a coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system after determining the coordinate conversion relationship according to the first point set, the second point set and the probability matrix;

[0063] a display module, configured to display the X-ray image projected by the projection module;

[0064] an adjustment module, configured to adjust the first point set based on an adjustment instruction of a user.

[0065] The determining module is further configured to determine a probability that each first position in the first point set adjusted by the adjusting module has a corresponding relationship with each second position in the second point set, and obtain an updated probability matrix; and determine an updated coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set, and the updated probability matrix.

[0066] Optionally, the adjusting module is specifically configured to:

[0067] receive an increase instruction of a user, and increase a first position specified by the user in the first point set;

[0068] receive a deletion instruction of a user, and delete a first position specified by the user in the first point set;

[0069] receive a modification instruction of a user, and modify a first position specified by the user in the first point set; and / or

[0070] receive a sequence adjustment instruction of a user, and adjust the sequence of each first position included in the first point set.

[0071] In a third aspect, an electronic device is provided, which includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory are in communication with each other through the communication bus.

[0072] The memory is configured to store a computer program.

[0073] The processor is configured to execute the program stored in the memory, and implement the method steps of any one of the first aspect.

[0074] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the method steps of any one of the first aspect are implemented.

[0075] In a fifth aspect, a computer program product including instructions, which, when executed on a computer, cause the computer to perform the automatic registration method of any one of the first aspect.

[0076] The embodiments of the present application have the following beneficial effects:

[0077] The automatic registration method, device, electronic device and medium provided by the embodiments of the present application can identify the first positions of the marker balls from the X-ray image, obtain a first point set, and obtain the second positions of each marker ball in the calibration plate coordinate system to obtain a second point set. Then, the probability that each first position and each second position have a corresponding relationship is determined to obtain a probability matrix, and then the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined according to the first point set, the second point set and the probability matrix. That is, the embodiments of the present application determine the coordinate conversion relationship based on the probability that the first position and the second position have a corresponding relationship, and do not need to determine the specific order of the first position, so that the problem of inaccurate registration caused by incorrect ordering can be eliminated, thereby improving the accuracy of determining the conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system. At the same time, there is no complex requirement for the size and arrangement of the steel balls, and the algorithm is robust and can well adapt to the case of partial misidentification of the position of the steel ball image, and is more widely applicable and more reliable.

[0078] Of course, implementing any product or method of the present application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can also be obtained by those skilled in the art based on these drawings.

[0080] Figure 1 A flow chart of an automatic registration method provided by the embodiments of the present application;

[0081] Figure 2 An exemplary schematic diagram of a display interface provided by the embodiments of the present application;

[0082] Figure 3 A flow chart of another automatic registration method provided by the embodiments of the present application;

[0083] Figure 4 An exemplary schematic diagram of another display interface provided by the embodiments of the present application;

[0084] Figure 5 A structural schematic diagram of an automatic registration device provided by the embodiments of the present application;

[0085] Figure 6 A structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0086] With reference to the drawings and embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art based on the present application shall fall within the scope of protection of the present application.

[0087] To improve the accuracy of determining the conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system, the present application provides an automatic registration method, which can be applied to an electronic device, for example, a surgical robot or a computer in communication connection with the surgical robot, and the present application does not make specific limitation to this. As shown in the figure, the method comprises the following steps: Figure 1

[0088] S101, obtaining an X-ray image taken of a calibration plate, and identifying a first position of each marker ball included in the calibration plate in an X-ray image coordinate system to obtain a first point set.

[0089] The calibration plate can be placed near a lesion, and then an X-ray image of the calibration plate and the lesion is taken, and then a digital image processing technology, for example, a SimpleBlob algorithm, is used to identify the marker balls from the X-ray image to obtain the first position of the center of each marker ball in the X-ray image coordinate system, and the first position of the center of each marker ball in the X-ray image coordinate system is a 2D position.

[0090] The X-ray image coordinate system is a two-dimensional coordinate system, and the coordinate axes included in the X-ray image coordinate system are all in the plane of the X-ray image, and the first position of the center of each marker ball in the X-ray image coordinate system is a 2D position.

[0091] S102, obtaining a second position of each marker ball in a calibration plate coordinate system to obtain a second point set.

[0092] The second position of the center of each marker ball in the calibration plate coordinate system can be obtained by three-coordinate instrument calibration to form the second point set. Alternatively, the second positions can also be obtained by other measurement methods, and the present application does not make specific limitation to this. The second position of the center of each marker ball in the calibration plate coordinate system is a 3D position.

[0093] The calibration plate coordinate system is a three-dimensional coordinate system, the center of the calibration plate can be taken as the coordinate origin, a vector passing through the coordinate origin and perpendicular to the calibration plate can be taken as one of the coordinate axes, and two mutually perpendicular vectors passing through the coordinate origin and parallel to the plane of the calibration plate can be taken as the other two coordinate axes to obtain the calibration plate coordinate system. Alternatively, the calibration plate coordinate system can be established by other methods, and the present application does not make specific limitation to this.

[0094] ​S103, determine a probability that each first position in the first point set and each second position in the second point set have a corresponding relationship, and obtain a probability matrix.

[0095] The first position and the second position having the corresponding relationship represent the same physical position. That is, each element in the probability matrix represents a probability that a physical position represented by a first position is the same as a physical position represented by a second position.

[0096] S104, determine a coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set, and the probability matrix.

[0097] Optionally, when the shooting device of the X-ray image belongs to a C-arm, the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system can also be referred to as a C-arm extrinsic parameter.

[0098] The automatic registration method provided in the embodiments of the present application can identify the first positions of the mark balls from the X-ray image, obtain the first point set, and obtain the second positions of each mark ball in the calibration plate coordinate system to obtain the second point set. Then, the probability that each first position and each second position have a corresponding relationship is determined to obtain the probability matrix, and then the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined according to the first point set, the second point set, and the probability matrix. That is, the coordinate conversion relationship is determined based on the probability that each first position and each second position have a corresponding relationship in the embodiments of the present application, and the specific order of the first positions does not need to be determined, so that the problem of inaccurate registration caused by incorrect ordering can be eliminated, thereby improving the accuracy of determining the conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system. The calculation method and process provided in the present patent do not have special requirements for the size of the steel ball and the arrangement mode of the steel ball, and the conversion relationship can also be calculated in the case that the small balls are partially blocked in the small ball identification process. The present application is more widely applicable, has better algorithm robustness, and has higher reliability.

[0099] In some embodiments of the present application, the manner of identifying the first position of each mark ball included in the calibration plate in the X-ray image coordinate system in S101 described above can be implemented as follows: converting the X-ray image into a binary image by using each preset threshold, wherein there are multiple preset thresholds, and thus multiple binary images are obtained. Extracting connected domains from each binary image, and merging the connected domains whose center points have a distance less than a preset distance. Then, screening the connected domains whose area, roundness, inertia ratio, and convexity parameters all meet the preset conditions, and taking the center point of each screened connected domain as a first position.

[0100] Compared with the method of extracting the marker ball position by using a fixed threshold, the embodiment of the application segments the binary image by using multiple thresholds, thereby being applicable to marker ball position recognition for different image gray scales, and improving the accuracy of determining the first point set.

[0101] The manner of determining the probability that each first position in the first point set corresponds to each second position in the second point set in S102 can be implemented as follows: for each first position included in the first point set and each second position included in the second point set, the second position is converted to the X-ray image coordinate system based on the initial coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system, to obtain a projection position, an error between the projection position and the first position is determined, and the probability that the first position and the second position have a corresponding relationship is obtained according to the determined error.

[0102] The initial coordinate conversion relationship can be randomly generated or obtained by other calibration methods, for example, by extracting calibration points from the X-ray image and sorting, and then by matching the sorted calibration points extracted from the X-ray image and the sorted calibration points extracted from the calibration plate coordinate system one by one and obtaining the initial coordinate conversion relationship based on the one-to-one matched calibration points. Although the coordinate conversion relationship obtained by other calibration methods has low accuracy, it has certain accuracy, so that the determined probability matrix has certain accuracy, which can accelerate the subsequent determination of a more accurate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system.

[0103] The probability that each first position and each second position have a corresponding relationship can be determined by the following formula (1) and formula (2):

[0104]

[0105]

[0106] wherein m ij is the probability that the ith first position and the jth second position have a corresponding relationship, exp represents an exponential function with e as the base, and α and β are both preset constants, R 0 is a rotation relationship of the initial coordinate conversion relationship, t 0 is a translation relationship of the initial coordinate conversion relationship, p i is the ith first position, y Ri is the jth second position.

[0107] Let the probability matrix be M=(m ij ), i=1,2,…n x , j=1,2,…n y .

[0108] In the embodiments of the present application, it is assumed that each first position and each second position can have a corresponding relationship, and therefore the relationship is represented by a probability matrix, and the probabilities of the same first position having a corresponding relationship with each second position in the probability matrix are different, thereby reflecting the corresponding relationship between the first point set and the second point set. It can be seen that the embodiments of the present application can obtain the corresponding relationship between the first point set and the second point set without sorting the first point set, thereby avoiding errors caused by sorting errors.

[0109] In some embodiments of the present application, before the S103 determines the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix, the electronic device can further perform row and column standardization on the probability matrix to obtain a row and column standardized probability matrix. When performing S103, the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system can be determined according to the first point set, the second point set and the row and column standardized probability matrix. The specific way of determining the coordinate conversion relationship can be referred to the subsequent description.

[0110] The row and column standardization of the probability matrix can be performed by the following formula (3) and formula (4):

[0111]

[0112]

[0113] wherein, m ij is the probability of the i th first position having a corresponding relationship with the j th second position, n x is the number of first positions included in the first point set, n y is the number of second positions included in the second point set.

[0114] After the row and column standardization of the probability matrix, the sum of each row element of the probability matrix is 1, and the sum of each column element is 1, which can more intuitively reflect the corresponding relationship between the first position and the second position, and can also accelerate the convergence speed of subsequent use of the probability matrix to solve the objective function.

[0115] The way of determining the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix in the above S103 can be implemented as follows: constructing an objective function according to the first point set, the second point set and the probability matrix, and then solving the objective function to obtain the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system when the objective function has a minimum value. Optionally, the probability matrix can be a probability matrix that has not been subjected to row and column standardization, or a probability matrix that has been subjected to row and column standardization.

[0116] The target function is used to determine the error between each projection position and each first position after each second position included in the second point set is projected to the X-ray image coordinate system based on the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system.

[0117] The target function can be formula (5):

[0118]

[0119] The target function value is E, n x is the number of first positions included in the first point set, n y is the number of second positions included in the second point set, m ij is the probability that the i-th first position and the j-th second position have a corresponding relationship, p i is the i-th first position, y Rj is the j-th second position, R k-1 is the rotation relationship of the coordinate conversion relationship obtained in the (k-1)th iteration in the solving process, t k-1 is the translation relationship of the coordinate conversion relationship obtained in the (k-1)th iteration in the solving process, k is the current iteration number, and α and β are both preset constants.

[0120] Optionally, the target function can be solved by using the least square method or the gradient descent method.

[0121] Taking the gradient descent method as an example, in the first iteration, k=1, m ij is obtained based on the initial coordinate conversion relationship, and is brought into formula (5). 2-1 The derivative of formula (5) is obtained, and R 2-1 and t ij are obtained in the derivative direction according to a preset step size. 2-1 In the second iteration, m 2-1 is updated based on the same way of formula (1) and formula (2), that is, R ij and t ij are brought into formula (6), and then the result of formula (6) is brought into formula (1) to obtain a new m 3-1 R 3-1 and t k are obtained by bringing the new m k into formula (5), deriving formula (5), and obtaining the derivative direction according to a preset step size.

[0122]

[0123] The target function value is E, n ijP(i, j) is a probability that there is a corresponding relationship between the ith first position and the jth second position, exp represents an exponential function with e as a base, and a and β are both preset constants, R k-1 R(k) is a rotation relationship of the coordinate conversion relationship obtained in the (k-1)th iteration in the solving process, t k-1 t(k) is a translation relationship of the coordinate conversion relationship obtained in the (k-1)th iteration in the solving process, k is a current iteration number, and p i xi is the ith first position, y Ri yj is the jth second position.

[0124] Optionally, in the iteration process, each time a new m ij is obtained, the new m ij may be subjected to row and column standardization, and then the row and column standardized m ij is brought into formula (5) to continue to solve the target function.

[0125] Through the above method, in the process of solving the target function, the coordinate conversion relationship and the probability matrix can be continuously optimized, so as to reduce the error of the probability matrix determined based on the initial coordinate conversion relationship, and the accuracy of determining the coordinate conversion relationship and the probability matrix is improved.

[0126] Due to the fact that the placement posture of the double-layer structure calibration plate is not parallel to the imaging surface of the X-ray imaging device, and the influence of ambient light and obstacles around the calibration plate, the marked balls in the X-ray image may be blocked or have uneven gray scale, thereby affecting the accuracy of identifying the first positions of the marked balls from the X-ray image. Therefore, the embodiments of the present application allow the user to adjust the first point set, and generate an optimized coordinate conversion relationship.

[0127] After the above S101 identifies the first positions of each marked ball included in the calibration plate in the X-ray image coordinate system to obtain the first point set, each first position can be marked when the X-ray image is displayed. Referring to Figure 2 , Figure 2The interface is for displaying an X-ray image. The left half of the interface is for displaying the X-ray image. The black dots in the X-ray image pointed to by marker 1 are all first positions. The area pointed to by marker 3 in the right half of the interface displays seven marker points in total. Each marker point represents a first position in the X-ray image. For example, marker point 1 (100.0, 200.5) represents a pixel point with coordinates (100.0, 200.5) in the X-ray image as a first position. When displaying the positions of the marker points, in addition to the coordinates, the radius of the connected domain in which the marker point is located can also be displayed, so as to facilitate the user to determine whether the connected domain corresponds to the marker ball. When detecting that the user selects a first position in the left half of the interface, the area pointed to by marker 2 in the lower area of the right half of the interface displays an enlarged image of the area in which the first position is located. When the user triggers the button pointed to by marker 6, the selected first position can be adjusted in the four directions of up, down, left and right. When the user triggers the button pointed to by marker 7, the magnification of the area in which the selected first position is located in the area pointed to by marker 2 can be reduced. When the user triggers the button pointed to by marker 8, the magnification of the area in which the selected first position is located in the area pointed to by marker 2 can be increased. When the user triggers the add marker button pointed to by marker 5, a first position can be added. The user can also trigger a delete marker button to delete the automatically recognized first position. The delete marker button is not shown in Figure 2

[0128] Referring to Figure 3 , after S103 of determining the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix, the electronic device can further perform the following steps:

[0129] S105, projecting each second position in the second point set onto the X-ray image based on the coordinate conversion relationship. The coordinate conversion relationship is obtained by solving the objective function.

[0130] For example, referring to Figure 4 , Figure 4 , markers 1-8 in Figure 2 have the same meanings as the corresponding markers, and marker 9 points to a white dot inside a black dot in the X-ray image, i.e., the projection position of each second position onto the X-ray image.

[0131] Optionally, after the projection, for each second position, a first position with the maximum probability corresponding to the second position can be determined from the latest probability matrix, and an error between the second position and the first position can be calculated. The average value of the errors obtained for each second position is taken as a reprojection error, and the reprojection error is displayed. For example, Figure 4 ​The area pointed by the mark 10 in the X-ray image shows the re-projection error of 0.01.

[0132] S106, display the projected X-ray image.

[0133] For example, refer to Figure 4 In the left half of the display interface, the X-ray image is displayed, and each first position included in the first point set is marked, and the projection position of each second position is also marked.

[0134] From the display interface, the user can observe the error between the projection position and the first position through the projected X-ray image, and thus intuitively understand the error of the coordinate conversion relationship. Moreover, the user can also observe the error of the coordinate conversion relationship through the numerical value of the re-projection error displayed on the display interface. Figure 4 As can be seen from the display interface, the user can observe the error between the projection position and the first position through the projected X-ray image, and thus intuitively understand the error of the coordinate conversion relationship. Moreover, the user can also observe the error of the coordinate conversion relationship through the numerical value of the re-projection error displayed on the display interface.

[0135] S107, adjust the first point set based on the adjustment instruction of the user.

[0136] Since the first positions are marked in the X-ray image, the user can conveniently observe the error between the automatically identified first positions and the marked ball center positions in the X-ray image; and the user can also observe the error of the coordinate conversion relationship through the numerical value of the re-projection error displayed on the display interface, so as to reduce the error by adjusting the first point set when any error is large.

[0137] Among them, the increase instruction of the user can be received to add the first position specified by the user to the first point set; the delete instruction of the user can be received to delete the first position specified by the user in the first point set; the modification instruction of the user can be received to modify the first position specified by the user in the first point set; and / or the order adjustment instruction of the user can be received to adjust the order of each first position included in the first point set. That is, the first point set can be added, deleted, position modified and order adjusted.

[0138] For example, refer to Figure 4 When it is detected that the user clicks the add mark button, the position selected or input by the user is added to the first point set as a new first position. When it is detected that the user selects one first position and clicks the delete mark button, the first position is deleted, wherein the delete mark button is not shown in Figure 4 When it is detected that the user drags the mark point in the area pointed by the mark 3, the order of each first position is updated to the order of the mark points after the dragging. When it is detected that the user selects one first position, an enlarged image of the area where the first position is located is displayed in the area pointed by the mark 2. When it is detected that the user clicks the button pointed by the mark 6, the first position is adjusted in the up, down, left and / or right four directions according to the user selection.

[0139] S108, determine a probability that each first position in the adjusted first point set has a corresponding relationship with each second position in the second point set, and obtain an updated probability matrix.

[0140] S109, determine an updated coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set, and the updated probability matrix.

[0141] Referring to Figure 4 After the user adjusts the first point set, if it is detected that the user clicks the re-registration button, S108 and S109 are executed, where a specific implementation of S108 can refer to S103 described above, and a specific implementation of S109 can refer to S104 described above.

[0142] Optionally, after the updated coordinate conversion relationship is determined in S109, the second point set can also be re-projected based on the updated coordinate conversion relationship, an updated re-projection error is calculated, and the projected X-ray image and the updated re-projection error are re-displayed.

[0143] In the related art, the angle between the marker points in the image can also be matched through a feature triangle template, and the marker points can be sorted. However, because the calibration plate with a double-layer structure and the imaging surface of the shooting device can form an arbitrary angle, the angle between the marker points on the X-ray image is also arbitrary, which makes it difficult to match through a template, and thus the accuracy of determining the sorting of the marker points is low and the robustness is poor. Moreover, because the marker points in the X-ray image can be blocked and have uneven grayscale, the marker points cannot be accurately extracted from the X-ray image, which leads to an error in the number of extracted calibration points and makes it impossible to determine the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system. If there is an error in the number or sorting of the marker points, the calibration plate and the lesion have to be shot again to obtain an X-ray image, which greatly increases the radiation dose of the patient.

[0144] In the embodiments of the present application, the coordinate conversion relationship and the probability matrix can be optimized in the process of solving the objective function to obtain a coordinate conversion relationship with the smallest projection error, thereby improving the accuracy of the coordinate conversion relationship. Moreover, the first point set can be manually adjusted, which improves the interactivity, reduces the error caused by missed detection and false detection of automatic identification of the first point set, and improves the robustness of determining the coordinate conversion relationship. Moreover, the coordinate conversion relationship can be optimized by manually adjusting the first point set in the embodiments of the present application, the X-ray registration success rate is high, that is, the accuracy of determining the coordinate conversion relationship is high, and the X-ray image does not need to be shot again, thereby reducing the radiation dose of the patient.

[0145] Based on the same inventive concept, corresponding to the method embodiments described above, the embodiments of the present application provide an automatic registration device, such as Figure 5As shown, the apparatus comprises an identification module 501, an acquisition module 502, and a determination module 503.

[0146] The identification module 501 is configured to acquire an X-ray image obtained by shooting a calibration plate, and identify a first position of each marker ball included in the calibration plate in an X-ray image coordinate system to obtain a first point set.

[0147] The acquisition module 502 is configured to acquire a second position of each marker ball in a calibration plate coordinate system to obtain a second point set.

[0148] The determination module 503 is configured to determine a probability that each first position in the first point set identified by the identification module 501 and each second position in the second point set acquired by the acquisition module 502 have a corresponding relationship, to obtain a probability matrix, wherein the first position and the second position having the corresponding relationship represent the same physical position.

[0149] The determination module 503 is further configured to determine a coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set, and the probability matrix.

[0150] The determination module 503 is specifically configured to:

[0151] For each first position included in the first point set and each second position included in the second point set, the determination module 503 converts the second position to the X-ray image coordinate system based on an initial coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system to obtain a projection position, determines an error between the projection position and the first position, and determines the probability that the first position and the second position have the corresponding relationship according to the determined error.

[0152] Optionally, the determination module 503 is specifically configured to:

[0153] The determination module 503 determines the probability that each first position and each second position have the corresponding relationship by the following formula:

[0154]

[0155]

[0156] wherein, m ij is the probability that the ith first position and the jth second position have the corresponding relationship, exp represents an exponential function with e as the base, α and β are both preset constants, R 0 is a rotation relationship of the initial coordinate conversion relationship, t 0 is a translation relationship of the initial coordinate conversion relationship, p i is the ith first position, and y Ri is the jth second position.

[0157] Optionally, the apparatus can further include:

[0158] a standardization module, configured to perform row and column standardization on the probability matrix to obtain a row and column standardized probability matrix, before determining the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix;

[0159] The determination module 503 is specifically configured to:

[0160] determine the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the row and column standardized probability matrix.

[0161] Optionally, the determination module 503 is specifically configured to:

[0162] construct a target function according to the first point set, the second point set and the probability matrix, wherein the target function is used to determine an error between each projection position obtained by projecting each second position included in the second point set to the X-ray image coordinate system and each first position based on the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system.

[0163] The coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined when the target function reaches a minimum value.

[0164] Optionally, the target function is:

[0165]

[0166] wherein E is a target function value, n x is a number of first positions included in the first point set, n y is a number of second positions included in the second point set, m ij is a probability that the ith first position and the jth second position have a corresponding relationship, p i is the ith first position, y Rj is the jth second position, R k-1 is a rotation relationship of the coordinate conversion relationship obtained in the (k-1)th iteration in the solving process, t k-1 is a translation relationship of the coordinate conversion relationship obtained in the (k-1)th iteration in the solving process, k is a current iteration number, and a and β are both preset constants.

[0167] Optionally, the apparatus can further include:

[0168] a projection module, configured to project each second position in the second point set to the X-ray image based on the coordinate conversion relationship after determining the coordinate conversion relationship between the calibration plate coordinate system and the X-ray image coordinate system according to the first point set, the second point set and the probability matrix.

[0169] The display module is used to display the X-ray image projected by the projection module.

[0170] The adjustment module is used to adjust the first point set based on the user's adjustment instructions;

[0171] The determination module 503 is also used to determine the probability that each first position in the first point set after adjustment by the adjustment module corresponds to each second position in the second point set, and to obtain the updated probability matrix; based on the first point set, the second point set and the updated probability matrix, the updated coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined.

[0172] Optional, adjust the module, specifically for:

[0173] Receive the user's add instruction and add the user-specified first position to the first point set;

[0174] Receive the user's deletion command and delete the first position specified by the user in the first point set;

[0175] Receive the user's modification instruction and modify the first position specified by the user in the first point set; and / or,

[0176] Receive the user's reordering instruction and adjust the order of the first positions included in the first point set.

[0177] This application also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0178] Memory 603 is used to store computer programs;

[0179] When the processor 601 executes the program stored in the memory 603, it implements the method steps in the above method embodiments.

[0180] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0181] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0182] The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0183] The aforementioned processor can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component.

[0184] In yet another embodiment provided in the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of any of the automatic registration methods described above.

[0185] In yet another embodiment provided in the present application, a computer program product containing instructions, which, when run on a computer, causes the computer to execute any of the automatic registration methods in the above embodiments.

[0186] In the embodiments described above, all or some of the steps can be implemented by hardware, software, firmware or any combination thereof. When implemented by software, all or some of the steps can be implemented in the form of one or more computer programs or program elements. The computer programs reside (at least temporarily) in a memory of a computer during execution. The memory can be a RAM memory, a flash memory, a ROM memory, an EPROM memory, or any other suitable memory. The memory can be integral to or separate from the computer. The computer programs can be written in any suitable programming language, such as C, C++, Java, Visual Basic, etc. The computer programs can be written in assembly or machine language, if desired. The computer programs can be distributed over network coupled file servers, or can be distributed by any other suitable means.

[0187] It is to be noted that the terms such as first and second, etc., are used herein merely to differentiate one entity or action from another, and do not necessarily require or imply any such actual relationship or order between such entities or actions. Also, the terms "comprising", "containing", or any other similar term are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not necessarily contain only those elements, but can contain other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0188] Each of the embodiments in the present specification is described in a related manner, and the same or similar parts among the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiments.

[0189] The above merely provides the preferred embodiment of the present application, and not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An automatic registration method, characterized in that, The method includes: Acquire X-ray images of the calibration plate and identify the first position of each marker ball included in the calibration plate in the X-ray image coordinate system to obtain a first point set; Obtain the second position of each marker ball in the calibration plate coordinate system to obtain the second point set; Determine the probability that each first position in the first point set corresponds to each second position in the second point set, and obtain a probability matrix, wherein the physical positions represented by the corresponding first and second positions are the same; Based on the first point set, the second point set, and the probability matrix, the coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined.

2. The method according to claim 1, characterized in that, The probability of determining a correspondence between each first position in the first point set and each second position in the second point set includes: For each first position included in the first point set and each second position included in the second point set, based on the initial coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system, the second position is transformed to the X-ray image coordinate system to obtain the projection position. The error between the projection position and the first position is determined, and the probability that the first position and the second position have a corresponding relationship is obtained based on the determined error.

3. The method according to claim 2, characterized in that, For each first position included in the first point set and each second position included in the second point set, based on the initial coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system, the second position is transformed to the X-ray image coordinate system to obtain the projected position. The error between the projected position and the first position is determined, and the probability that there is a correspondence between the first position and the second position is obtained based on the determined error, including: The probability of a correspondence between each first position and each second position is determined by the following formula: Where, m ij Let be the probability that the i-th first position and the j-th second position correspond, exp represent an exponential function with base e, α and β are preset constants, and Rij is the probability that there is a correspondence between them. 0 The rotation relationship of the initial coordinate transformation relationship, t 0 p represents the translation relationship of the initial coordinate transformation relationship. i For the i-th first position, y Ri It represents the j-th second position.

4. The method according to claim 2 or 3, characterized in that, Before determining the coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system based on the first point set, the second point set, and the probability matrix, the method further includes: The probability matrix is ​​standardized by row and column to obtain the standardized probability matrix. Determining the coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system based on the first point set, the second point set, and the probability matrix includes: Based on the first point set, the second point set, and the row- and column-normalized probability matrix, the coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined.

5. The method according to any one of claims 1-3, characterized in that, Determining the coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system based on the first point set, the second point set, and the probability matrix includes: Based on the first point set, the second point set, and the probability matrix, an objective function is constructed; wherein, the objective function is used to determine the error between each projected position and each first position after projecting each second position included in the second point set onto the X-ray image coordinate system based on the coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system; The coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system when the objective function is minimized.

6. The method according to claim 5, characterized in that, The objective function is: Where E is the objective function value, n x n is the number of the first positions included in the first point set. y m is the number of second positions included in the second point set. ij Let p be the probability that there is a correspondence between the i-th first position and the j-th second position. i For the i-th first position, y Rj For the j-th second position, R k-1 To determine the rotation relationship of the coordinate transformation obtained in the (k-1)th iteration during the solution process, t k-1 This is the translation relationship of the coordinate transformation relationship obtained in the (k-1)th iteration during the solution process, where k is the current iteration number, and α and β are preset constants.

7. The method according to any one of claims 1-3, characterized in that, After determining the coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system based on the first point set, the second point set, and the probability matrix, the method further includes: Based on the coordinate transformation relationship, each second position in the second point set is projected onto the X-ray image; Display the projected X-ray image; Adjust the first point set based on the user's adjustment instructions; Determine the probability that each first position in the adjusted first point set corresponds to each second position in the second point set, and obtain the updated probability matrix; Based on the first point set, the second point set, and the updated probability matrix, the updated coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system is determined.

8. The method according to claim 7, characterized in that, The adjustment of the first point set based on the user's adjustment instructions includes: Receive the user's add instruction and add the first position specified by the user to the first point set; Receive the user's deletion command and delete the first position specified by the user in the first point set; Receive the user's modification instruction and modify the first position specified by the user in the first point set; and / or, The system receives a reordering instruction from the user and adjusts the order of the first positions included in the first point set.

9. An automatic registration device, characterized in that, The device includes: The identification module is used to acquire X-ray images of the calibration plate and identify the first position of each marker ball included in the calibration plate in the X-ray image coordinate system to obtain a first point set; The acquisition module is used to obtain the second position of each marker ball in the calibration plate coordinate system, thus obtaining the second point set; The determining module is used to determine the probability that each first position in the first point set identified by the identification module corresponds to each second position in the second point set acquired by the acquisition module, and obtains a probability matrix, wherein the physical positions represented by the corresponding first and second positions are the same. The determining module is further configured to determine the coordinate transformation relationship between the calibration plate coordinate system and the X-ray image coordinate system based on the first point set, the second point set, and the probability matrix.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

Images