A distortion calibration method and device, electronic equipment and medium

Abstract

This invention provides a distortion calibration method, apparatus, electronic device, and medium, relating to the field of image processing technology, which can more accurately calibrate the distortion of X-ray images. The invention includes: acquiring an X-ray image to be calibrated, which is an image obtained by scanning multiple Mark spheres with X-rays using a C-arm. Then, the center pixel of each Mark sphere in the X-ray image to be calibrated is determined, and the physical center point corresponding to each center pixel is determined. The physical center point is the position of the center of the Mark sphere in the flange coordinate system. Next, each pair of center pixels and physical centers is matched to obtain the distortion parameters of the C-arm. The distortion parameters are the coefficients of each monomial in a polynomial fitting curve, which represents the transformation relationship between the flange coordinate system and the X-ray image coordinate system. Finally, based on the distortion parameters of the C-arm, distortion calibration is performed on the X-ray image to be calibrated.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a distortion calibration method, apparatus, electronic device, and medium. Background Technology

[0002] In applications such as machine vision, image registration, and 3D reconstruction, distortion calibration of X-ray images taken by a C-arm X-ray machine is necessary to accurately register X-ray and CT images, determine the relationship between the pixel coordinate system and the image coordinate system of the X-ray image, and eliminate the impact of X-ray image distortion on the surgical navigation robot's positioning accuracy.

[0003] The existing distortion calibration method is the Zhang Zhengyou calibration method, which is a method for designing distortion calibration for images captured by a camera. The main forms of lens distortion in cameras are radial distortion and tangential distortion. Because the imaging principle of a C-arm differs from that of a camera, the forms of lens distortion also differ. For example, C-arm distortion includes pincushion distortion (a type of radial distortion), S-shaped distortion caused by magnetic fields, and local deformation distortion caused by structural weight and other factors. Therefore, using the Zhang Zhengyou calibration algorithm to calibrate the distortion of X-ray images captured by a C-arm is insufficient to eliminate distortions other than radial distortion, leading to inaccurate distortion calibration of X-ray images. Summary of the Invention

[0004] The purpose of this invention is to provide a distortion calibration method, apparatus, electronic device, and medium to more accurately calibrate X-ray images for distortion correction. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of the present invention provide a distortion calibration method, the method comprising:

[0006] Acquire an X-ray image to be calibrated, wherein the X-ray image to be calibrated is an image obtained by scanning multiple Mark spheres with X-rays using a C-arm;

[0007] Determine the center pixel of each Mark sphere in the X-ray image to be calibrated;

[0008] Determine the physical point of the sphere corresponding to each sphere center pixel, where the physical point of the sphere center is the position of the center of the Mark sphere in the flange coordinate system;

[0009] For each pair of sphere center pixels and sphere center physical points, the distortion parameters of the C-arm are obtained; wherein, the distortion parameters are the coefficients of each monomial in the polynomial fitting curve, and the polynomial fitting curve is used to represent the transformation relationship between the flange coordinate system and the X-ray image coordinate system.

[0010] Based on the distortion parameters of the C-arm, the X-ray image to be calibrated is calibrated for distortion.

[0011] Optionally, the process of matching each pair of sphere center pixels and sphere center physical points to obtain the distortion parameters of the C-arm includes:

[0012] For each pair of sphere center pixels and sphere center physical points, based on the transformation relationship between the X-ray image coordinate system and the C-arm camera coordinate system, the first projection point of the sphere center pixel in the C-arm camera coordinate system is determined;

[0013] Based on the transformation relationship between the flange coordinate system and the C-arm camera coordinate system, the second projection point of the sphere's center physical point in the C-arm camera coordinate system is determined;

[0014] Substituting the first projection point and the second projection point into the polynomial fitting curve, the target equation is obtained; wherein, the target equation is used to represent the residual between the sphere center pixel and the sphere center projection point;

[0015] Construct the Jacobian matrix based on each objective equation;

[0016] Using the LM algorithm, the intrinsic parameters, extrinsic parameters, and distortion parameters of the C-arm are determined based on the Jacobian matrix when the residual between the center pixel and the center physical point is minimized.

[0017] Optionally, the polynomial fitting curve includes:

[0018] x″=a0+a1x′+a2y′+a3x′ 2 +a4x′y′+a5y′ 2 +a6x′ 3 +a7x′ 2 y′+a8x′y′ 2 +a9y′ 3 +a 10 x′ 4 +a 11 x′ 3 y′+a 12 x′ 2 y′ 2 +a 13 x′y′ 3 +a 14 y′ 4

[0019] y″=b0+b1x′+b2y′+b3x′ 2 +b4x′y′+b5y′ 2 +b6x′ 3 +b7x′ 2 y′+b8x′y′ 2+b9y′ 3 +b 10 x′ 4 +b 11 x′ 3 y′+b 12 x′ 2 y′ 2 +b 13 x′y′ 3 +b 14 y′ 4

[0020] Where, u = f x *x″+c x v = f y *y″+c y x″ is the coordinate of the sphere's center pixel on the X-axis of the C-arm camera coordinate system, x′ is the coordinate of the sphere's center physical point on the X-axis of the C-arm camera coordinate system, y″ is the coordinate of the sphere's center pixel on the Y-axis of the C-arm camera coordinate system, y′ is the coordinate of the sphere's center physical point on the Y-axis of the C-arm camera coordinate system, a0~a 14 and b0~b 14 The distortion parameter is defined as follows: u is the coordinate on the horizontal axis of the X-ray image coordinate system, v is the coordinate on the vertical axis of the X-ray image coordinate system, and f is the distortion parameter. x c x f y cy and cy are the intrinsic parameters of the C-arm.

[0021] Optionally, when using the LM algorithm to determine the minimum residual between the center pixel and the center physical point of the sphere based on the Jacobian matrix, the intrinsic parameters, extrinsic parameters, and distortion parameters of the C-arm include:

[0022] Update the intrinsic parameters, extrinsic parameters, and distortion parameters using the following formula:

[0023] x k+1 -x k =-(J T J+μI) -1 J T f(x k )

[0024] Where J is the Jacobian matrix, T denotes the transpose of the matrix, μ is the damping factor, I is the identity matrix, and x k To update the parameter vector consisting of the intrinsic, extrinsic, and distortion parameters of the C-arm before the update, x k+1 Let f(x) be the parameter vector consisting of the updated intrinsic, extrinsic, and distortion parameters of the C-arm. k Let ) be the objective equation, and k be the number of iterations;

[0025] If the currently calculated parameter vector does not meet the iteration termination condition, then return to the step of updating the intrinsic parameters, extrinsic parameters and the distortion parameters using the following formula;

[0026] If the currently calculated parameter vector satisfies the iteration termination condition, then the currently calculated parameter vector is used as the intrinsic, extrinsic, and distortion parameters of the C-arm.

[0027] Secondly, embodiments of the present invention provide a distortion calibration device, the device comprising:

[0028] The acquisition module is used to acquire the X-ray image to be calibrated, which is an image obtained by scanning multiple Mark balls with X-rays using a C-arm;

[0029] The determination module is used to determine the center pixel of each Mark ball in the X-ray image to be calibrated;

[0030] The determining module is also used to determine the physical point of the ball center corresponding to each ball center pixel, wherein the physical point of the ball center is the position of the center of the Mark ball in the flange coordinate system;

[0031] The matching module is used to match each pair of sphere center pixels and sphere center physical points to obtain the distortion parameters of the C-arm; wherein, the distortion parameters are the coefficients of each monomial in the polynomial fitting curve, and the polynomial fitting curve is used to represent the transformation relationship between the flange coordinate system and the X-ray image coordinate system.

[0032] The calibration module is used to perform distortion calibration on the X-ray image to be calibrated based on the distortion parameters of the C-arm.

[0033] Optionally, the matching module is specifically used for:

[0034] For each pair of sphere center pixels and sphere center physical points, based on the transformation relationship between the X-ray image coordinate system and the C-arm camera coordinate system, the first projection point of the sphere center pixel in the C-arm camera coordinate system is determined;

[0035] Based on the transformation relationship between the flange coordinate system and the C-arm camera coordinate system, the second projection point of the sphere's center physical point in the C-arm camera coordinate system is determined;

[0036] Substituting the first projection point and the second projection point into the polynomial fitting curve, the target equation is obtained; wherein, the target equation is used to represent the residual between the sphere center pixel and the sphere center projection point;

[0037] Construct the Jacobian matrix based on each objective equation;

[0038] Using the LM algorithm, the intrinsic parameters, extrinsic parameters, and distortion parameters of the C-arm are determined based on the Jacobian matrix when the residual between the center pixel and the center physical point is minimized.

[0039] Optionally, the polynomial fitting curve includes:

[0040] x″=a0+a1x′+a2y′+a3x′ 2 +a4x′y′+a5y′ 2 +a6x′ 3 +a7x′ 2 y′+a8x′y′ 2 +a9y′ 3 +a 10 x′ 4 +a 11 x′ 3 y′+a 12 x′ 2 y′ 2 +a 13 x′y′ 3 +a 14 y′ 4

[0041] y″=b0+b1x′+b2y′+b3x′ 2 +b4x′y′+b5y′ 2 +b6x′ 3 +b7x′ 2 y′+b8x′y′ 2 +b9y′ 3 +b 10 x′ 4 +b 11 x′ 3 y′+b 12 x′ 2 y′ 2 +b 13 x′y′ 3 +b 14 y′ 4

[0042] Where, u = f x *x″+c x v = f y *y″+c y x″ is the coordinate of the sphere's center pixel on the X-axis of the C-arm camera coordinate system, x′ is the coordinate of the sphere's center physical point on the X-axis of the C-arm camera coordinate system, y″ is the coordinate of the sphere's center pixel on the Y-axis of the C-arm camera coordinate system, y′ is the coordinate of the sphere's center physical point on the Y-axis of the C-arm camera coordinate system, a0~a 14 and b0~b 14The distortion parameter is defined as follows: u is the coordinate on the horizontal axis of the X-ray image coordinate system, v is the coordinate on the vertical axis of the X-ray image coordinate system, and f is the distortion parameter. x c x f y and c y This is the internal parameter of the C-arm.

[0043] Optionally, the matching module is specifically used for:

[0044] Update the intrinsic parameters, extrinsic parameters, and distortion parameters using the following formula:

[0045] x k+1 -x k =-(J T J+μI) -1 J T f(x k )

[0046] Where J is the Jacobian matrix, T denotes the transpose of the matrix, μ is the damping factor, I is the identity matrix, and x k To update the parameter vector consisting of the intrinsic, extrinsic, and distortion parameters of the C-arm before the update, x k+1 Let f(x) be the parameter vector consisting of the updated intrinsic, extrinsic, and distortion parameters of the C-arm. k Let ) be the objective equation, and k be the number of iterations;

[0047] If the currently calculated parameter vector does not meet the iteration termination condition, then return to the step of updating the intrinsic parameters, extrinsic parameters and the distortion parameters using the following formula;

[0048] If the currently calculated parameter vector satisfies the iteration termination condition, then the currently calculated parameter vector is used as the intrinsic, extrinsic, and distortion parameters of the C-arm.

[0049] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0050] Memory, used to store computer programs;

[0051] When a processor executes a program stored in memory, it implements the steps of any of the above distortion calibration methods.

[0052] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described distortion calibration methods.

[0053] Fifthly, embodiments of the present invention also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the above-described distortion calibration methods.

[0054] The distortion calibration method, apparatus, electronic device, and medium provided in this invention do not consider specific distortion types when performing distortion calibration. Instead, they consider the relationship between the center pixel and the center physical point in the X-ray image. Since the center physical point is the position of the center of the Mark sphere in the flange coordinate system, i.e., the physical location of the center of the Mark sphere, and the relationship between the center pixel and its physical location is the result of various distortion types, this invention is equivalent to considering various types of distortion. Therefore, the distortion calibration of X-ray images performed by this invention is more accurate.

[0055] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0057] Figure 1 This is a flowchart of a distortion calibration method provided in an embodiment of the present invention;

[0058] Figure 2 A flowchart illustrating a method for determining the distortion parameters of a C-arm, as provided in an embodiment of the present invention;

[0059] Figure 3 This is a flowchart of another distortion calibration method provided in an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of the structure of a distortion calibration device provided in an embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of the present invention.

[0063] In related technologies, the Zhang Zhengyou calibration method is generally used to calibrate the distortion of images captured by the camera.

[0064] The camera model for Zhang Zhengyou's calibration method is:

[0065]

[0066]

[0067] x′=x / z (3)

[0068] y′=y / z (4)

[0069]

[0070]

[0071] Where r 2 =x′ 2 +y′ 2 (7)

[0072] u = f x *x″+c x (8)

[0073] v = f y *y″+c y (9)

[0074] Where s is a preset scale factor, for example s = 1, u is the coordinate on the horizontal axis of the image coordinate system, v is the coordinate on the vertical axis of the image coordinate system, and f x c x f y and c y r is the camera's intrinsic parameter. 11 ~r 33 t1~t3 are the extrinsic parameters of the camera, i.e., r 11 ~r 33 Let t1 be the rotation parameter from the world coordinate system to the camera coordinate system, t3 be the translation parameters from the world coordinate system to the camera coordinate system, X, Y, and Z represent the coordinates on the three axes of the world coordinate system, x, y, and z represent the coordinates on the three axes of the camera coordinate system, R′ be the rotation parameter from the world coordinate system to the camera coordinate system, and t be the translation parameter from the world coordinate system to the camera coordinate system. Represents the radial distortion of the X-axis of the camera coordinate system, 2p1x′y′+p2(r 2 +2x′ 2 ) represents the tangential distortion along the Y-axis of the camera coordinate system. 2p1(r) represents the radial distortion of the X-axis of the camera coordinate system. 2 +2y′2 )+p2x′y′ represents the tangential distortion of the Y-axis of the camera coordinate system, and k1~k6, p1 and p2 are distortion parameters. Where represents the constraint condition, that is, formula (7) is the constraint condition of formula (6).

[0075] This shows that Zhang Zhengyou's calibration method considers both radial and tangential distortion. However, the imaging principle of a C-arm differs from that of a camera, resulting in different forms of lens distortion. For example, C-arm distortion includes pincushion distortion (a type of radial distortion), S-shaped distortion caused by magnetic fields, and localized deformation distortion caused by structural weight and other factors. Therefore, using Zhang Zhengyou's calibration algorithm to calibrate X-ray images captured by a C-arm cannot eliminate distortions other than radial distortion, leading to inaccurate distortion calibration of the X-ray images.

[0076] To more accurately calibrate X-ray images for distortion correction, this invention provides a distortion correction method. This method can be applied to electronic devices, such as computers, servers, or tablets, which possess image processing capabilities. Figure 1 As shown, the distortion calibration method provided in this embodiment of the invention includes the following steps:

[0077] S101, acquire the X-ray image to be calibrated.

[0078] The X-ray image to be calibrated is an image obtained by scanning multiple Mark spheres with X-rays using a C-arm.

[0079] In one embodiment, multiple Mark spheres can be embedded in a Mark plate, which is placed between the X-ray emitter of the C-arm and the image intensifier of the C-arm. The X-ray image to be calibrated is then captured using the X-ray arm. The image intensifier is used to increase the brightness of the X-ray image captured by the C-arm.

[0080] S102, determine the center pixel of each Mark sphere in the X-ray image to be calibrated.

[0081] In one implementation, circles in the X-ray image to be calibrated can be identified, and the center of each circle in the X-ray image to be calibrated can be used as a sphere center pixel.

[0082] S103, determine the physical point of the sphere center corresponding to each sphere center pixel. The physical point of the sphere center is the position of the center of the Mark sphere in the flange coordinate system.

[0083] In this embodiment of the invention, the physical position of each sphere's center pixel is the same as the position of its corresponding sphere's center physical point.

[0084] In this embodiment of the invention, a Mark plate embedded with multiple Mark balls is mounted on a front-end instrument of a C-arm robotic arm. The robotic arm drags the Mark plate between the X-ray transmitter and the image intensifier of the C-arm for imaging, obtaining an X-ray image to be calibrated. Therefore, the X-ray image to be calibrated includes the image of the front-end instrument. When executing S103, the image of the front-end instrument can be used as a reference to match the physical point of the center of each ball's center pixel.

[0085] The position of the center of each Mark sphere in the flange coordinate system, i.e., the physical point of the center of the sphere, can be obtained by calibration using a high-precision calibration instrument. The flange coordinate system is a world coordinate system that can reflect the actual physical position.

[0086] S104: Match each pair of sphere center pixels and sphere center physical points to obtain the distortion parameters of the C-arm.

[0087] The distortion parameter is the coefficient of each monomial in the polynomial fitting curve, which is used to represent the transformation relationship between the flange coordinate system and the X-ray image coordinate system.

[0088] S105 performs distortion calibration on the X-ray image to be calibrated based on the distortion parameters of the C-arm.

[0089] The distortion calibration method provided in this embodiment of the invention does not consider specific distortion types when performing distortion calibration. Instead, it considers the relationship between the center pixel and the center physical point in the X-ray image. Since the center physical point is the position of the center of the Mark sphere in the flange coordinate system, that is, the physical position of the center of the Mark sphere, and the relationship between the center pixel and its physical position is the result of various distortion types, this embodiment of the invention is equivalent to considering various types of distortion. Therefore, the distortion calibration of X-ray images performed by this embodiment of the invention is more accurate.

[0090] See Figure 2 In this embodiment of the invention, the method of matching each pair of sphere center pixels and sphere center physical points in S104 to obtain the distortion parameters of the C-arm can be implemented by the following steps:

[0091] S1041, for each pair of sphere center pixels and sphere center physical points, based on the transformation relationship between the X-ray image coordinate system and the C-arm camera coordinate system, determine the first projection point of the sphere center pixel in the C-arm camera coordinate system.

[0092] S1042, based on the transformation relationship between the flange coordinate system and the C-arm camera coordinate system, determine the second projection point of the physical point at the center of the sphere in the C-arm camera coordinate system.

[0093] S1043, Substitute the first projection point and the second projection point into the polynomial fitting curve to obtain the target equation.

[0094] The objective equation is used to represent the relationship between the intrinsic parameters, extrinsic parameters, and distortion parameters of the C-arm.

[0095] S1044, construct the Jacobian matrix based on each objective equation.

[0096] S1045 uses the LM algorithm to determine the intrinsic, extrinsic, and distortion parameters of the C-arm when the residual between the center pixel and the center physical point is minimized based on the Jacobian matrix.

[0097] The C-arm camera model and polynomial model provided in this embodiment of the invention are shown in formulas (10) to (17). The following describes the above-mentioned C-arm camera model and polynomial model in conjunction with the C-arm camera model and polynomial model provided in this embodiment of the invention. Figure 2 Explanation:

[0098]

[0099]

[0100] x′=x / z (12)

[0101] y′=y / z (13)

[0102] x″=a0+a1x′+a2y′+a3x′ 2 +a4x′y′+a5y′ 2 +a6x′ 3 +a7x′ 2 y′+a8x′y′ 2 +a9y′ 3 +a 10 x′ 4 +a 11 x′ 3 y′+a 12 x′ 2 y′ 2 +a 13 x′y′ 3 +a 14 y′ 4 (14)

[0103] y″=b0+b1x′+b2y′+b3x′ 2 +b4x′y′+b 5y ′ 2 +b6x′ 3 +b7x′ 2 y′+b8x′y′ 2 +b9y′ 3 +b 10 x′ 4 +b 11x′ 3 y′+b 12 x′ 2 y′ 2 +b 13 x′y′ 3 +b 14 y′ 4 (15)

[0104] u = f x *x″+c x (16)

[0105] v = f y *y″+c y (17)

[0106] Where s is a preset scale factor, for example s = 1, u is the coordinate on the horizontal axis of the X-ray image coordinate system, v is the coordinate on the vertical axis of the X-ray image coordinate system, and f x c x f y and c y For the intrinsic parameters of the C-arm, r 11 ~r 33 t1~t3 are the extrinsic parameters of the C-arm, i.e., r 11 ~r 33 Let t1 be the rotation parameter from the flange coordinate system to the C-arm camera coordinate system, t1 to t3 be the translation parameters from the flange coordinate system to the C-arm camera coordinate system, X, Y, and Z represent the coordinates on the three axes of the flange coordinate system, x, y, and z represent the coordinates on the three axes of the C-arm camera coordinate system, R′ be the rotation parameter from the flange coordinate system to the C-arm camera coordinate system, t be the translation parameter from the flange coordinate system to the C-arm camera coordinate system, and a0 to a 14 and b0~b 14 This is the distortion parameter.

[0107] The aforementioned extrinsic parameters of the C-arm refer to the rotation parameters (r) from the flange coordinate system to the C-arm camera coordinate system. 11 ~r 33 And the translation parameters (t1~t3). That is, r 11 ~r 33 Let R′ be the integer part of the integer part, and t1~t3 be the integer part of the integer part.

[0108] When executing S1041 above, for each pair of sphere center pixels and sphere center physical points, the coordinates of the sphere center pixels are substituted into the above formulas (16) and (17) to obtain x″=(uc x ) / f x and y″=(vc y ) / f yAt this point, x″ is the coordinate of the center pixel on the X-axis of the C-arm camera coordinate system, and y″ is the coordinate of the center pixel on the Y-axis of the C-arm camera coordinate system.

[0109] When executing S1042, the coordinates of the physical point of the sphere corresponding to the sphere center pixel of S1041 are substituted into the above formulas (11), (12) and (13) to obtain x′ and y′. At this time, x′ is the coordinate of the physical point of the sphere center on the X-axis of the C-arm camera coordinate system, and y′ is the coordinate of the physical point of the sphere center on the Y-axis of the C-arm camera coordinate system.

[0110] When performing S1043 above, the first projection point and the second projection point are substituted into the polynomial fitting curve, that is, substituted into the above formula (14) and formula (15) to obtain the target equation.

[0111] In one embodiment, the x″ obtained in S1041 is substituted into the left side of the equation (14), and the x′ obtained in S1042 is substituted into the right side of the equation (14) to obtain a0~a 14 f x c x The relationship equation between R' and t is f(a, fx, cx, R', t).

[0112] Meanwhile, substituting y″ obtained from S1041 into the left side of equation (15), and substituting y′ obtained from S1042 into the right side of equation (15), we obtain b0~b 14 f y c y The relationship equation between R' and t is f(b, fy, cy, R', t).

[0113] Then, by combining f(a, fx, cx, R', t) and f(b, fy, cy, R', t), we obtain the objective equation of the form f(a, b, C, R). Here, a represents a0 to a... 14 b is b0~b 14 C is f x c x f y and c y That is, the intrinsic parameter of the C-arm, R is r 11 ~r 33 And t1~t3, which are the extrinsic parameters of the C-arm. Understandably, a target equation can be obtained based on each pair of sphere center pixels and sphere center physics.

[0114] When executing S1044, a, b, C, and R are taken as independent variables, and partial derivatives are obtained with respect to f(a, b, C, R) to obtain the Jacobian matrix. The Jacobian matrix is ​​given by formula (18):

[0115]

[0116] Where J is the Jacobian matrix, f1 is the target equation f1(a, b, C, R) obtained based on the first pair of sphere center pixels and sphere center physical points, and so on, f m The objective equation f is obtained based on the m-th pair of center pixels and center physical points. m (a, b, C, R), where m is the number of point pairs of center pixels and center physical points participating in the matching. m can be the total number of point pairs of center pixels and center physical points determined in S103, or m can be a specified number. In this case, when S104 is executed, a specified number of center pixels and center physical points are selected from the pairs of center pixels and center physical points determined in S103. n is 4, x1 is a, x2 is b, x3 is C, and x4 is R.

[0117] When executing S1045, the Levenberg-Marquarelt (LM) algorithm is used, that is, using (J T J+μI)Δx=-J T Iterative optimization of f yields the global optimal solution, which represents the intrinsic, extrinsic, and distortion parameters of the C-arm. Here, J is the Jacobian matrix, T denotes the matrix transpose, μ is the damping factor, and I is the identity matrix. Δx=x k+1 -x k This refers to the residual of the independent variables, where x1 is a, x2 is b, x3 is C, and x4 is R. It represents the residual between the value of the parameter vector (x1 to x4) after one iteration and the value before the iteration. f represents f1(a, b, C, R) to f... m (a, b, C, R).

[0118] In this embodiment of the invention, the iterative process using the LM algorithm in S1045 is as follows:

[0119] Update the intrinsic, extrinsic, and distortion parameters of the C-arm using the following formulas:

[0120] x k+1 -x k =-(J T J+μI) -1 J T f(x k (19)

[0121] Where J is the Jacobian matrix, T denotes the transpose of the matrix, μ is the damping factor, I is the identity matrix, and x k To update the parameter vector consisting of the intrinsic, extrinsic, and distortion parameters of the C-arm before the update, x k+1Let f(x) be the parameter vector consisting of the updated intrinsic, extrinsic, and distortion parameters of the C-arm. k ) represents the objective equation, and k represents the number of iterations.

[0122] If the currently calculated parameter vector does not meet the iteration termination condition, return to the step of updating the intrinsic, extrinsic, and distortion parameters using the following formula.

[0123] If the currently calculated parameter vector satisfies the iteration termination condition, then the currently calculated parameter vector will be used as the intrinsic, extrinsic, and distortion parameters of the C-arm.

[0124] Since the LM algorithm model is (J T J+μI)Δx=-J T f(x k Therefore, the iterative formula is Δx = x k+1 -x k =-(J T J+μI) -1 J T f(x k Based on Zhang Zhengyou's calibration method, the intrinsic, extrinsic, and distortion parameters can be calculated, and the calculated intrinsic, extrinsic, and distortion parameters can be used to form the initial value of the parameter vector, namely x0. Based on the initial value of the parameter vector, the parameter vector is updated using the iterative formula (19), and it is determined whether the iteration termination condition is met. If not, the value of the parameter vector is updated based on the currently calculated parameter vector until the iteration termination condition is met. Then, the parameter values ​​included in the currently calculated parameter vector are taken as the global optimal solution (intrinsic, extrinsic, and distortion parameters of the C-arm). Among them, μ can be adjusted according to the iteration speed.

[0125] For example, iteration termination conditions include: the number of iterations reaches a preset number, and / or (x k+1 -x k < preset difference.

[0126] Since the LM algorithm is based on the Gauss-Newton algorithm, it is easier to solve nonlinear problems than the least squares method. Furthermore, the LM algorithm incorporates a damping factor μ, which allows for more flexible determination of the global optimum while avoiding getting trapped in local optima. Therefore, compared to using the least squares method to determine the distortion parameters, C-arm intrinsic parameters, and extrinsic parameters, the distortion parameters, C-arm intrinsic parameters, and extrinsic parameters determined in this embodiment of the invention are more accurate.

[0127] See Figure 3 The following describes the overall process of the distortion calibration method provided in the embodiments of the present invention, in conjunction with application scenarios:

[0128] S301, obtain the input physical points of each sphere's center.

[0129] Among them, the physical point of the center of the ball is the position of each Mark ball on the Mark plate in the flange coordinate system. The position of the Mark ball in the flange coordinate system can be obtained by calibration using a high-precision calibration instrument.

[0130] S302 uses a C-arm robotic arm to move a Mark plate mounted on the front end of the robotic arm between the X-ray transmitter and the image intensifier of the C-arm for imaging, thereby obtaining multiple X-ray images.

[0131] In this embodiment of the invention, when the C-arm takes X-ray images, the pose of the X-ray images can be changed by the robotic arm, so that the C-arm can take pictures of the Mark plate from different angles and obtain multiple X-ray images.

[0132] S303, determine the sphere center pixel in each X-ray image, and determine the sphere center physical point corresponding to each sphere center pixel.

[0133] The method for determining the center pixel in S303 is the same as in S102 above, and can be referred to the description in S102 above, so it will not be repeated here. The method for determining the physical center point of each center pixel in S303 is the same as in S103 above, and can be referred to the description in S103 above, so it will not be repeated here.

[0134] S304. Using Zhang Zhengyou's calibration method, based on multiple pairs of sphere center pixels and sphere center physical points, the initial intrinsic and extrinsic parameters of the C-arm are obtained.

[0135] Calculating the initial intrinsic and extrinsic parameters can reduce the number of iterations required to determine the subsequent intrinsic and extrinsic parameters, thereby improving the efficiency of distortion calibration.

[0136] S305, for each pair of sphere-centered pixels and sphere-centered physical points in each X-ray image to be calibrated, based on the transformation relationship between the X-ray image coordinate system and the C-arm camera coordinate system, determine the first projection point of the sphere-centered pixel in the C-arm camera coordinate system.

[0137] Each X-ray image in the series can be used as an X-ray image to be calibrated.

[0138] The method for determining the first projection point in S305 is the same as that in S1041 above, and can be referred to the description in S1041 above, which will not be repeated here.

[0139] S306. Based on the transformation relationship between the flange coordinate system and the C-arm camera coordinate system, determine the second projection point of the physical point at the center of the sphere in the C-arm camera coordinate system.

[0140] The method for determining the second projection point in S306 is the same as that in S1042 above, and can be referred to the description in S1042 above, which will not be repeated here.

[0141] S307, Substitute the first projection point and the second projection point into the polynomial fitting curve to obtain the target equation.

[0142] The objective equation is used to represent the relationship between the intrinsic parameters, extrinsic parameters, and distortion parameters of the C-arm.

[0143] The method for determining the target equation in S307 is the same as that in S1043 above, and can be referred to the description in S1043 above, which will not be repeated here.

[0144] S308, construct the Jacobian matrix based on each objective equation.

[0145] The method for constructing the Jacobian matrix in S308 is the same as that in S1044 above, and can be referred to the description in S1044 above, which will not be repeated here.

[0146] S309 uses the LM algorithm to determine the intrinsic, extrinsic, and distortion parameters of the C-arm when the residual between the center pixel and the center physical point is minimized based on the Jacobian matrix.

[0147] The method for determining the intrinsic, extrinsic, and distortion parameters of the C-arm in S309 is the same as that in S1045 above, and can be referred to the description in S1045 above, which will not be repeated here.

[0148] S310 performs distortion calibration on the X-ray image to be calibrated based on the distortion parameters of the C-arm.

[0149] In one embodiment, after determining the distortion parameters, the coordinates of each pixel in the X-ray image to be calibrated are converted to (x″, y″) using the above formulas (16) and (17), and then converted to (x′, y′) using formulas (14) and (15). (x′, y′) are then used as the pixel coordinates after distortion calibration.

[0150] Based on the same inventive concept, and corresponding to the above method embodiments, this invention provides a distortion calibration device, such as... Figure 4 As shown, the device includes: an acquisition module 401, a determination module 402, a matching module 403, and a calibration module 404;

[0151] The acquisition module 401 is used to acquire the X-ray image to be calibrated, which is an image obtained by scanning multiple Mark balls with X-rays using a C-arm;

[0152] The determination module 402 is used to determine the center pixel of each Mark ball in the X-ray image to be calibrated;

[0153] The determination module 402 is also used to determine the physical point of the ball center corresponding to each ball center pixel, where the physical point of the ball center is the position of the center of the Mark ball in the flange coordinate system;

[0154] The matching module 403 is used to match each pair of sphere center pixels and sphere center physical points to obtain the distortion parameters of the C-arm; wherein, the distortion parameters are the coefficients of each monomial in the polynomial fitting curve, and the polynomial fitting curve is used to represent the transformation relationship between the flange coordinate system and the X-ray image coordinate system.

[0155] The calibration module 404 is used to perform distortion calibration on the X-ray image to be calibrated based on the distortion parameters of the C-arm.

[0156] Optional, matching module 403, specifically used for:

[0157] For each pair of sphere center pixels and sphere center physical points, based on the transformation relationship between the X-ray image coordinate system and the C-arm camera coordinate system, the first projection point of the sphere center pixel in the C-arm camera coordinate system is determined;

[0158] Based on the transformation relationship between the flange coordinate system and the C-arm camera coordinate system, the second projection point of the physical point at the center of the sphere in the C-arm camera coordinate system is determined;

[0159] Substituting the first and second projection points into the polynomial fitting curve yields the target equation; where the target equation represents the residual between the sphere center pixel and the sphere center projection point.

[0160] Construct the Jacobian matrix based on each objective equation;

[0161] Using the LM algorithm, the intrinsic, extrinsic, and distortion parameters of the C-arm are determined based on the Jacobian matrix when the residual between the center pixel and the center physical point is minimized.

[0162] Optionally, the polynomial fitting curve includes:

[0163] x″=a0+a1x′+a2y′+a3x′ 2 +a4x′y′+a5y′ 2 +a6x′ 3 +a7x′ 2 y′+a8x′y′ 2 +a9y′ 3 +a 10 x′ 4 +a 11 x′ 3 y′+a 12 x′ 2 y′ 2 +a 13 x′y′ 3 +a 14 y′ 4

[0164] y″=b0+b1x′+b2y′+b3x′ 2 +b4x′y′+b5y′ 2 +b6x′ 3 +b7x′ 2 y′+b8x′y′ 2 +b9y′ 3 +b 10 x′ 4 +b 11 x′ 3 y′+b 12 x′ 2 y′ 2 +b 13 x′y′ 3 +b 14 y′ 4

[0165] Where, u = f x *x″+c x v = f y *y″+c y x″ is the coordinate of the sphere's center pixel on the X-axis of the C-arm camera coordinate system, x′ is the coordinate of the sphere's center physical point on the X-axis of the C-arm camera coordinate system, y″ is the coordinate of the sphere's center pixel on the Y-axis of the C-arm camera coordinate system, y′ is the coordinate of the sphere's center physical point on the Y-axis of the C-arm camera coordinate system, a0~a 14 and b0~b 14 Here, u is the coordinate on the horizontal axis of the X-ray image coordinate system, v is the coordinate on the vertical axis of the X-ray image coordinate system, and f is the distortion parameter. x c x f y and c y This is the internal reference for the C-arm.

[0166] Optional, matching module 403, specifically used for:

[0167] Update the intrinsic, extrinsic, and distortion parameters using the following formulas:

[0168] x k+1 -x k =-(J T J+μI) -1 J T f(x k )

[0169] Where J is the Jacobian matrix, T denotes the transpose of the matrix, μ is the damping factor, I is the identity matrix, and x k To update the parameter vector consisting of the intrinsic, extrinsic, and distortion parameters of the C-arm before the update, x k+1 Let f(x) be the parameter vector consisting of the updated intrinsic, extrinsic, and distortion parameters of the C-arm.k Let ) be the objective equation, and k be the number of iterations;

[0170] If the currently calculated parameter vector does not meet the iteration termination condition, return to the step of updating the intrinsic, extrinsic, and distortion parameters using the following formula;

[0171] If the currently calculated parameter vector satisfies the iteration termination condition, then the currently calculated parameter vector will be used as the intrinsic, extrinsic, and distortion parameters of the C-arm.

[0172] This invention also provides an electronic device, such as... Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503, and a communication bus 504. The processor 501, communication interface 502, and memory 503 communicate with each other via the communication bus 504.

[0173] Memory 503 is used to store computer programs;

[0174] When the processor 501 executes the program stored in the memory 503, it implements the method steps in the above method embodiments.

[0175] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0176] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0177] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0178] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0179] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described distortion calibration methods.

[0180] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the distortion calibration methods described above.

[0181] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0182] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0183] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0184] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A distortion calibration method, characterized by, The method includes: An X-ray image to be calibrated is acquired, which is an image obtained by scanning multiple Mark balls with X-rays using a C-arm; a Mark plate with multiple Mark balls embedded is mounted on the front-end instrument of the C-arm robotic arm; Determine the center pixel of each Mark sphere in the X-ray image to be calibrated; Determine the physical point of the sphere corresponding to each sphere center pixel, where the physical point of the sphere center is the position of the center of the Mark sphere in the flange coordinate system; For each pair of sphere-centered pixel points and sphere-centered physical points, based on the transformation relationship between the X-ray image coordinate system and the C-arm camera coordinate system, the first projection point of the sphere-centered pixel point in the C-arm camera coordinate system is determined; based on the transformation relationship between the flange coordinate system and the C-arm camera coordinate system, the second projection point of the sphere-centered physical point in the C-arm camera coordinate system is determined; the first projection point and the second projection point are substituted into a polynomial fitting curve to obtain the target equation; the target equation is used to represent the residual between the sphere-centered pixel point and the sphere-centered projection point; based on each target equation, a Jacobian matrix is ​​constructed; using the LM algorithm, based on the Jacobian matrix, the intrinsic parameters, extrinsic parameters, and distortion parameters of the C-arm are determined when the residual between the sphere-centered pixel point and the sphere-centered physical point is minimized; wherein, the distortion parameters are the coefficients of each monomial in the polynomial fitting curve, and the polynomial fitting curve is used to represent the transformation relationship between the flange coordinate system and the X-ray image coordinate system; Based on the distortion parameters of the C-arm, the X-ray image to be calibrated is calibrated for distortion.

2. The method of claim 1, wherein, The polynomial fitting curve includes: ; ; in, , , Let be the coordinates of the sphere's center pixel on the X-axis of the C-arm camera coordinate system. Let be the coordinates of the physical point at the center of the sphere on the X-axis of the C-arm camera coordinate system. Let be the Y-coordinate of the sphere's center pixel in the C-arm camera coordinate system. Let be the coordinates of the physical point at the center of the sphere on the Y-axis of the C-arm camera coordinate system. ~ and ~ The distortion parameter is... Let v be the coordinate on the horizontal axis of the X-ray image coordinate system, and let v be the coordinate on the vertical axis of the X-ray image coordinate system. , , and This is the internal parameter of the C-arm.

3. The method of claim 1, wherein, The intrinsic parameters, extrinsic parameters, and distortion parameters of the C-arm, when determined using the LM algorithm based on the Jacobian matrix to minimize the residual between the center pixel and the center physical point, include: Update the intrinsic parameters, extrinsic parameters, and distortion parameters using the following formula: x k+1 - x k = -1 ; in, Let be the Jacobian matrix. Represents the transpose of a matrix. The damping factor, Let x be the identity matrix. k To update the parameter vector consisting of the intrinsic, extrinsic, and distortion parameters of the C-arm before the update, x k+1 This is the parameter vector consisting of the intrinsic, extrinsic, and distortion parameters of the updated C-arm. Let k be the objective equation, and k be the number of iterations. If the currently calculated parameter vector does not meet the iteration termination condition, then return to the step of updating the intrinsic parameters, extrinsic parameters and the distortion parameters using the following formula; If the currently calculated parameter vector satisfies the iteration termination condition, then the currently calculated parameter vector is used as the intrinsic, extrinsic, and distortion parameters of the C-arm.

4. A distortion calibration apparatus characterized by comprising: The device includes: An acquisition module is used to acquire an X-ray image to be calibrated, which is an image obtained by scanning multiple Mark balls with X-rays by a C-arm; a Mark plate with multiple Mark balls embedded is mounted on the front-end instrument of the C-arm robotic arm; The determination module is used to determine the center pixel of each Mark ball in the X-ray image to be calibrated; The determining module is also used to determine the physical point of the ball center corresponding to each ball center pixel, wherein the physical point of the ball center is the position of the center of the Mark ball in the flange coordinate system; The matching module is specifically used for: for each pair of sphere-centered pixel points and sphere-centered physical points, based on the transformation relationship between the X-ray image coordinate system and the C-arm camera coordinate system, determining the first projection point of the sphere-centered pixel point in the C-arm camera coordinate system; based on the transformation relationship between the flange coordinate system and the C-arm camera coordinate system, determining the second projection point of the sphere-centered physical point in the C-arm camera coordinate system; substituting the first projection point and the second projection point into a polynomial fitting curve to obtain a target equation; the target equation is used to represent the residual between the sphere-centered pixel point and the sphere-centered projection point; constructing a Jacobian matrix based on each target equation; using the LM algorithm, determining the intrinsic parameters, extrinsic parameters, and distortion parameters of the C-arm when the residual between the sphere-centered pixel point and the sphere-centered physical point is minimized based on the Jacobian matrix; wherein, the distortion parameters are the coefficients of each monomial in the polynomial fitting curve, and the polynomial fitting curve is used to represent the transformation relationship between the flange coordinate system and the X-ray image coordinate system; The calibration module is used to perform distortion calibration on the X-ray image to be calibrated based on the distortion parameters of the C-arm.

5. The apparatus according to claim 4, characterized in that, The polynomial fitting curve includes: ; ; in, , , Let be the coordinates of the sphere's center pixel on the X-axis of the C-arm camera coordinate system. Let be the coordinates of the physical point at the center of the sphere on the X-axis of the C-arm camera coordinate system. Let be the Y-coordinate of the sphere's center pixel in the C-arm camera coordinate system. Let be the coordinates of the physical point at the center of the sphere on the Y-axis of the C-arm camera coordinate system. ~ and ~ The distortion parameter is... Let v be the coordinate on the horizontal axis of the X-ray image coordinate system, and let v be the coordinate on the vertical axis of the X-ray image coordinate system. , , and This is the internal parameter of the C-arm.

6. The apparatus of claim 4, wherein, The matching module is specifically used for: Update the intrinsic parameters, extrinsic parameters, and distortion parameters using the following formula: x k+1 - x k = -1 ; in, Let be the Jacobian matrix. To represent the transpose of a matrix, The damping factor, Let x be the identity matrix. k To update the parameter vector consisting of the intrinsic, extrinsic, and distortion parameters of the C-arm before the update, x k+1 This is the parameter vector consisting of the intrinsic, extrinsic, and distortion parameters of the updated C-arm. Let k be the objective equation, and k be the number of iterations. If the currently calculated parameter vector does not meet the iteration termination condition, then return to the step of updating the intrinsic parameters, extrinsic parameters and the distortion parameters using the following formula; If the currently calculated parameter vector satisfies the iteration termination condition, then the currently calculated parameter vector is used as the intrinsic, extrinsic, and distortion parameters of the C-arm.

7. An electronic device, comprising: It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-3.

Images