An ultrasonic probe calibration method and device based on data dimension reduction and homographic transformation

Abstract

The application discloses an ultrasonic probe calibration method and device based on data dimension reduction and homographic transformation, and relates to the technical field of ultrasonic probe calibration. The method comprises the following steps: firstly, calculating the two-dimensional coordinates of each calibration point on an ultrasonic image based on an N-line calibration tool, and converting the two-dimensional coordinates to the ultrasonic probe coordinate system; then, determining a calibration plane coordinate system based on the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system, and determining the transformation matrix between the two; finally, projecting the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system into the calibration plane, and calculating the homographic transformation matrix; storing the calculated transformation matrix and homographic transformation matrix, and subsequently, conveniently and accurately converting the two-dimensional coordinates of any target point on the ultrasonic image into the three-dimensional coordinates of the target point in the ultrasonic probe coordinate system according to the transformation matrix and the homographic transformation matrix. The application can simplify the existing calibration process and improve the calibration accuracy of the ultrasonic probe.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of ultrasonic probe calibration, in particular to an ultrasonic probe calibration method and device based on data dimension reduction and homographic transformation. BACKGROUND

[0002] An ultrasonic detection device is a common detection device in medicine and is used for perspective subcutaneous tissue and has the characteristics of real-time and no radiation. An ultrasonic probe is the core of the ultrasonic detection device, converts electric energy into ultrasonic energy through a piezoelectric crystal, contacts the human body, returns to the ultrasonic vibrator, is converted into an electric signal, and is converted into a two-dimensional black and white image. Ultrasonic three-dimensional reconstruction is an important way to expand the field of view of doctors, and the distortion of an ultrasonic image leads to that the information provided by a single two-dimensional picture is very limited. At present, the mainstream research direction is to perform three-dimensional reconstruction on a scanning region through continuous ultrasonic sequences, so as to realize the effect of CT scanning and guide doctors or robots to perform surgery.

[0003] However, the spatial position relationship between the ultrasonic sequences is a prerequisite for realizing three-dimensional reconstruction, and currently, three main methods are constraint method, image method and tracking method. The tracking method is the most commonly used ultrasonic three-dimensional reconstruction method, and the additional positioning devices used include optical tracking, electromagnetic tracking, accelerometer and the like. For the tracking method, the position calibration between the ultrasonic probe and the positioning device (referred to as ultrasonic probe calibration) is a prerequisite. However, most of the existing ultrasonic probe calibration tools have problems such as complicated process and low precision.

[0004] Therefore, it is necessary to provide a new calibration tool for an ultrasonic probe calibration method to simplify the existing calibration process and improve the calibration precision. SUMMARY

[0005] The application aims to provide an ultrasonic probe calibration method and device based on data dimension reduction and homographic transformation, which can simplify the existing calibration process and improve the calibration precision of the ultrasonic probe.

[0006] To achieve the above-mentioned purpose, the application provides the following solutions.

[0007] In a first aspect, the application provides an ultrasonic probe calibration method based on data dimension reduction and homographic transformation, comprising the following steps:

[0008] A plurality of calibration points are selected on an ultrasonic image obtained by the ultrasonic probe, and two-dimensional coordinates of the calibration points on the ultrasonic image are calculated based on an N-line calibration tool. The N-line calibration tool is a three-layer N-line calibration tool based on binary coding.

[0009] The two-dimensional coordinates of each calibration point are converted to the ultrasonic probe coordinate system based on the pose relationship among the binocular camera, the ultrasonic probe and the N-line calibration tool, so as to obtain the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system; the visual markers are mounted on the ultrasonic probe and the N-line calibration tool, and the pose of the ultrasonic probe and the N-line calibration tool can be tracked and acquired by the binocular camera.

[0010] Based on the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system, the calibration plane coordinate system is determined, and based on the calibration plane coordinate system and the ultrasonic probe coordinate system, the transformation matrix is determined.

[0011] According to the transformation matrix, the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system are projected into the calibration plane, and the homographic transformation matrix is calculated according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasonic image.

[0012] The transformation matrix and the homographic transformation matrix are stored, and the calibration of the ultrasonic probe is completed; according to the transformation matrix and the homographic transformation matrix, the two-dimensional coordinates of any target point on the ultrasonic image can be converted into the three-dimensional coordinates of the target point in the ultrasonic probe coordinate system.

[0013] Optionally, the two-dimensional coordinates of any calibration point on the ultrasonic image are calculated according to the following formula:

[0014]

[0015] wherein q i is the two-dimensional coordinates of the i-th calibration point on the ultrasonic image, q i-1 and q i+1 are the two-dimensional coordinates of two calibration points adjacent to the i-th calibration point on the ultrasonic image; p i and p i+1 are the two-dimensional coordinates of the i-th end point and the i+1-th end point of the N-line calibration tool on the ultrasonic image.

[0016] Optionally, the two-dimensional coordinates of each calibration point are converted to the ultrasonic probe coordinate system according to the following formula:

[0017]

[0018] wherein q i is the two-dimensional coordinates of the i-th calibration point on the ultrasonic image, m i is the three-dimensional coordinates of the i-th calibration point in the ultrasonic probe coordinate system, is the relative pose of the ultrasonic probe coordinate system in the binocular camera coordinate system, is the relative pose of the N-line calibration tool in the binocular camera coordinate system.

[0019] Optionally, based on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system, a calibration plane coordinate system is determined, and based on the calibration plane coordinate system and the ultrasound probe coordinate system, a transformation matrix is determined, specifically including the following steps:

[0020] The least square method is used to perform plane fitting on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system to obtain a calibration plane equation.

[0021] The calibration plane equation is converted into a least square matrix, and the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are substituted into the least square matrix to obtain a normal vector of the calibration plane.

[0022] A plane vector is constructed based on any two points in the calibration plane and the normal vector of the calibration plane.

[0023] Based on the calibration plane coordinate system and the ultrasound probe coordinate system, a transformation matrix is determined.

[0024] Optionally, the calibration plane equation is as shown in the following formula:

[0025] Ax+By+Cz+D=0 (C≠0).

[0026] Wherein, A, B, C, D are coefficients of the calibration plane equation, and (x, y, z) are three-dimensional coordinates of any calibration point in the ultrasound probe coordinate system.

[0027] The calibration plane equation can be deformed into the following formula:

[0028] z=ax+by+c.

[0029] Wherein, are intermediate custom coefficients.

[0030] The least square matrix is as shown in the following formula:

[0031] b=AX.

[0032] Wherein, (x i ,y i ,z i ) are three-dimensional coordinates of the i-th calibration point in the ultrasound probe coordinate system, and i takes a value in the range of [1, n].

[0033] Optionally, the transformation matrix is represented by the following formula:

[0034]

[0035] Wherein, M pR is a rotation matrix of the calibration plane coordinate system, t is a translation vector of the origin of the calibration plane coordinate system, t = -Rl1, and l1 is the origin of the calibration plane coordinate system.

[0036] Optionally, the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected into the calibration plane according to the following formula:

[0037]

[0038] where (x mi ,y mi ,z mi ) is the three-dimensional coordinates of the i-th calibration point in the ultrasound probe coordinate system, (x si ,y si ) is the two-dimensional coordinates of the i-th calibration point on the calibration plane, k i is the depth value of the i-th calibration point on the calibration plane, and k i is 0 in theory.

[0039] Optionally, the homographic transformation matrix is calculated according to the following formula:

[0040]

[0041] where α is a scale transformation factor, M h is the homographic transformation matrix, and (x oi ,y oi ) is the two-dimensional coordinates of the i-th calibration point on the ultrasound image.

[0042] Optionally, the two-dimensional coordinates of any target point on the ultrasound image are converted into the three-dimensional coordinates of the target point in the ultrasound probe coordinate system according to the transformation matrix and the homographic transformation matrix, specifically including:

[0043] The two-dimensional coordinates of any target point on the ultrasound image are obtained, and the two-dimensional coordinates of the target point on the calibration plane are calculated according to the two-dimensional coordinates of the target point on the ultrasound image and the homographic transformation matrix.

[0044] The three-dimensional coordinates of the target point in the ultrasound probe coordinate system are calculated according to the two-dimensional coordinates of the target point on the calibration plane and the transformation matrix.

[0045] In a second aspect, the application provides an ultrasound probe calibration device based on data dimension reduction and homographic transformation, which comprises a binocular camera, an ultrasound probe, an N-line calibration tool, and a data processing device; visual markers are installed on the ultrasound probe and the N-line calibration tool, and the poses of the ultrasound probe and the N-line calibration tool can be tracked and obtained by using the binocular camera; the N-line calibration tool is a three-layer N-line calibration tool based on binary coding.

[0046] The data processing device is configured to: select a plurality of calibration points on an ultrasound image acquired by an ultrasound probe, and calculate two-dimensional coordinates of each of the calibration points on the ultrasound image based on an N-line calibration tool; convert the two-dimensional coordinates of each of the calibration points to an ultrasound probe coordinate system based on a pose relationship among the binocular camera, the ultrasound probe, and the N-line calibration tool, to obtain three-dimensional coordinates of each of the calibration points in the ultrasound probe coordinate system; determine a calibration plane coordinate system based on the three-dimensional coordinates of each of the calibration points in the ultrasound probe coordinate system, and determine a transformation matrix based on the calibration plane coordinate system and the ultrasound probe coordinate system; project the three-dimensional coordinates of each of the calibration points in the ultrasound probe coordinate system into the calibration plane according to the transformation matrix, and calculate a homographic transformation matrix based on the two-dimensional coordinates of each of the calibration points on the calibration plane and the two-dimensional coordinates of each of the calibration points on the ultrasound image; store the transformation matrix and the homographic transformation matrix, to complete calibration of the ultrasound probe; and convert two-dimensional coordinates of any target point on the ultrasound image into three-dimensional coordinates of the target point in the ultrasound probe coordinate system according to the transformation matrix and the homographic transformation matrix.

[0047] According to specific embodiments provided in the present application, the following technical effects are disclosed:

[0048] The present application provides an ultrasound probe calibration method and device based on data dimension reduction and homographic transformation, which comprises: selecting a plurality of calibration points on an ultrasound image acquired by an ultrasound probe, and calculating two-dimensional coordinates of each of the calibration points on the ultrasound image based on an N-line calibration tool; converting the two-dimensional coordinates of each of the calibration points to an ultrasound probe coordinate system based on a pose relationship among the binocular camera, the ultrasound probe, and the N-line calibration tool, and then determining a calibration plane coordinate system based on the three-dimensional coordinates of each of the calibration points in the ultrasound probe coordinate system, and determining a transformation matrix based on the calibration plane coordinate system and the ultrasound probe coordinate system; projecting the three-dimensional coordinates of each of the calibration points in the ultrasound probe coordinate system into the calibration plane according to the transformation matrix, and calculating a homographic transformation matrix; storing the calculated transformation matrix and homographic transformation matrix, to complete calibration of the ultrasound probe; and subsequently, conveniently and accurately converting two-dimensional coordinates of any target point on the ultrasound image into three-dimensional coordinates of the target point in the ultrasound probe coordinate system according to the transformation matrix and the homographic transformation matrix. The scheme provided in the present application fits an ultrasound imaging plane, reduces three-dimensional points to two-dimensional points using the plane, and uses homographic transformation to represent the relationship between each of the two-dimensional points after dimension reduction and each point on the ultrasound image, cancels the scale factor used in the current mainstream calibration tool, avoids the problem of amplification of calibration error caused by scale factor estimation error, and can simplify the existing calibration process while improving the calibration accuracy of the ultrasound probe. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed in the embodiments will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained based on these drawings without creative labor.

[0050] Figure 1 A flow chart of an ultrasonic probe calibration method based on data dimension reduction and homographic transformation provided by an embodiment of the present application.

[0051] Figure 2 A schematic diagram of an N-line calibration tool in an ultrasonic probe calibration method based on data dimension reduction and homographic transformation provided by an embodiment of the present application.

[0052] Figure 3 A flow chart of step A3 in an ultrasonic probe calibration method based on data dimension reduction and homographic transformation provided by an embodiment of the present application.

[0053] Figure 4 A functional module schematic diagram of an ultrasonic probe calibration device based on data dimension reduction and homographic transformation provided by an embodiment of the present application.

[0054] Figure 5 A schematic diagram of an ultrasonic probe calibration scene built in an ultrasonic probe calibration method based on data dimension reduction and homographic transformation provided by another embodiment of the present application.

[0055] Figure 6 A schematic diagram of a calibration tool in an ultrasonic probe calibration method based on data dimension reduction and homographic transformation provided by another embodiment of the present application.

[0056] Figure 7 A schematic diagram of a calibration point interface picked up in an ultrasonic probe calibration method based on data dimension reduction and homographic transformation provided by another embodiment of the present application.

[0057] Figure 8 A result schematic diagram of a calibration point completely picked up in an ultrasonic probe calibration method based on data dimension reduction and homographic transformation provided by another embodiment of the present application.

[0058] Figure 9 A result schematic diagram of a calibration point not completely picked up in an ultrasonic probe calibration method based on data dimension reduction and homographic transformation provided by another embodiment of the present application.

[0059] Figure 10 A schematic diagram of a set of calibration contrast experiment result curves in an ultrasonic probe calibration method based on data dimension reduction and homographic transformation provided by another embodiment of the present application.

[0060] Figure 11 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0061] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0062] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] In one exemplary embodiment, such as Figure 1 The flowchart shown illustrates an ultrasonic probe calibration method based on data dimensionality reduction and homography transformation, comprising the following steps:

[0064] A1. Select several calibration points on the ultrasound image acquired by the ultrasound probe, and calculate the two-dimensional coordinates of each calibration point on the ultrasound image based on the N-line calibration fixture; the N-line calibration fixture is a three-layer N-line calibration fixture based on binary encoding.

[0065] The traditional N-line model only has three line segments, such as Figure 2 middle Assuming there are intersection points q0, q1, and q2 in the ultrasound image, only point q1 can have its coordinates calculated directly using the N-line model principle. Therefore, the utilization rate of the calibration point is only 1 / 3.

[0066] To increase the number and utilization of calibration points, and improve calibration accuracy while reducing the number of calibration images required, this embodiment extends the N-line model, designing a three-layer 24-line model. Each calibration plane contains eight calibration line segments (solid lines). The dashed lines represent the intersections of the ultrasound imaging plane and the calibration plane, with intersection points ranging from q0 to q... 23 Thus, assuming all calibration points are identified, q1~q6, q9~q 14 q 17 ~q 22 The coordinates of all 18 points can be directly calculated using the N-line model, achieving a calibration point utilization rate of 3 / 4.

[0067] Based on the principle of similar triangles, removing the points at both ends that cannot be calculated, for q1~q6, q9~q 14 q 17 ~q 22 In all cases, the following relationship holds:

[0068]

[0069] wherein, is a proportional relationship in three-dimensional space, which can be represented by the pixel distance on the ultrasound image:

[0070]

[0071] According to the vector operation law:

[0072]

[0073] In the actual production process of the model, the calibration line segments are also encoded by the thickness of the lines, so as to facilitate identification, thin lines represent 0, and thick lines represent 1. From the left column, they are {000, 001, 010, …, 111} in turn, which represent the numbers 0-7 in decimal, which is also the reason why 8 line segments are used for each calibration plane.

[0074] In an exemplary embodiment, the two-dimensional coordinates of any calibration point on the ultrasound image are calculated according to the following formula in step A1:

[0075]

[0076] wherein q i is the two-dimensional coordinates of the i-th calibration point on the ultrasound image, q i-1 and q i+1 are the two-dimensional coordinates of the two calibration points adjacent to the i-th calibration point on the ultrasound image; p i and p i+1 are the two-dimensional coordinates of the i-th end point and the i+1-th end point of the N-line calibration tool on the ultrasound image, respectively.

[0077] A2, based on the pose relationship between the binocular camera, the ultrasound probe and the N-line calibration tool, the two-dimensional coordinates of each calibration point are converted to the ultrasound probe coordinate system to obtain the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system; the visual marker is mounted on the ultrasound probe and the N-line calibration tool, and the pose of the ultrasound probe and the N-line calibration tool can be tracked and obtained by the binocular camera.

[0078] Let the pose of the N-line calibration tool under the binocular camera be the pose of the ultrasound probe under the camera be Let the coordinates of the calibration point under the ultrasound probe coordinate system be m i =[x mi ,y mi ,z mi ] TAccording to the relationship of the spatial coordinate system, in the embodiment, the two-dimensional coordinates of each calibration point can be converted to the ultrasound probe coordinate system according to the following formula:

[0079]

[0080] wherein q i is the two-dimensional coordinate of the i-th calibration point on the ultrasound image, m i is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, is the relative pose of the ultrasound probe coordinate system in the binocular camera coordinate system, is the relative pose of the N-line calibration tool in the binocular camera coordinate system.

[0081] A3, based on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system, determine the calibration plane coordinate system, and based on the calibration plane coordinate system and the ultrasound probe coordinate system, determine the transformation matrix.

[0082] Since the ultrasound imaging is a plane, all points are theoretically in the same plane in the ultrasound probe coordinate system, so a plane-plane homography matrix can be used to represent the mapping relationship of the two-dimensional coordinates of the image to the three-dimensional coordinates in the ultrasound probe coordinate system. However, the three-dimensional coordinates in the ultrasound probe coordinate system need to be converted to two-dimensional coordinates first. Therefore, it is considered to reduce the dimension by projecting the points to the plane in the ultrasound probe coordinate system.

[0083] The least squares method is used to perform plane fitting on the three-dimensional coordinates in the ultrasound probe coordinate system, and the general form of the plane equation is:

[0084] Ax+By+Cz+D=0(C≠0).

[0085] which is transformed into:

[0086]

[0087] Let that is:

[0088] z=ax+by+c.

[0089] which can be converted to the least squares matrix b=AX, wherein

[0090] Substituting the three-dimensional coordinates of the points in the ultrasound probe coordinate system, a, b, and c can be obtained, and the normal vector of the calibration plane can be obtained.

[0091] In an exemplary embodiment, as shown in the flowchart of Figure 3 Step A3 specifically includes the following steps:

[0092] A31, the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are fitted by a least square method to obtain a calibration plane equation. The calibration plane equation is shown in the following formula:

[0093] Ax+By+Cz+D=0 (C≠0).

[0094] Wherein, A, B, C, D are coefficients of the calibration plane equation, (x, y, z) are three-dimensional coordinates of any calibration point in the ultrasound probe coordinate system.

[0095] The calibration plane equation can be deformed into the following formula:

[0096] z=ax+by+c.

[0097] Wherein, are intermediate custom coefficients.

[0098] A32, the calibration plane equation is converted into a least square matrix, and the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are substituted to obtain the normal vector of the calibration plane. In this embodiment, the least square matrix is shown in the following formula:

[0099] b=AX.

[0100] Wherein, (x i ,y i ,z i ) is the three-dimensional coordinates of the i-th calibration point in the ultrasound probe coordinate system, and i is in the range of [1, n].

[0101] A33, two points in the calibration plane are selected to form a plane vector, and based on the plane vector and the normal vector of the calibration plane, a calibration plane coordinate system is constructed.

[0102] Next, a coordinate system is established in the plane, and two points l1 and l2 are substituted as shown in the following formula:

[0103]

[0104] Let From the vector product formula can be solved Then the rotation matrix of the new coordinate system is shown in the following formula:

[0105]

[0106] Then, an arbitrary point is specified as the origin, and it is assumed that l1, so the translation vector is shown in the following formula:

[0107] t=-Rl1.

[0108] S34, based on the calibration plane coordinate system and the ultrasound probe coordinate system, determine the transformation matrix. The transformation matrix is as follows:

[0109]

[0110] Wherein, M p is the transformation matrix, R is the rotation matrix of the calibration plane coordinate system, t is the translation vector of the origin of the calibration plane coordinate system, t = -Rl1, l1 is the origin of the calibration plane coordinate system.

[0111] A4, according to the transformation matrix, the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected into the calibration plane, and the homographic transformation matrix is calculated according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasound image.

[0112] Through the transformation matrix M p All three-dimensional points can be projected into a two-dimensional calibration plane, and the points s i ={x si ,y si} T That is, there is the following relationship:

[0113]

[0114] Wherein, k i The value should be zero in the theoretical case, but due to the existence of error, it is often not zero, and it is directly discarded.

[0115] At this time, the projection of three-dimensional coordinates to two-dimensional calibration plane is completed, and then the homographic transformation matrix M h of the two-dimensional point set {o} to {s} needs to be calculated, assuming that there is the following relationship:

[0116]

[0117] Wherein, α is the scale transformation factor, and after expansion and elimination of α, the following formula is obtained:

[0118]

[0119] After deformation, the following formula can be obtained:

[0120]

[0121] Further can be transformed into the least square matrix Ax = 0, wherein:

[0122]

[0123] x = [h 11 h 12 h 13h 21 h 22 h 23 h 31 h 32 h 33 ] T .

[0124] Ax=0 is an over-determined equation, and there is no exact solution, so it is estimated by SVD decomposition. Assuming that there is a set of x that makes Ax=0 true, then x multiplied by any proportional coefficient k obviously still holds true, so it is not difficult to increase the constraint ||x||=1, so as to construct an optimization problem with constraints, as shown in the following formula:

[0125] argmin||Ax||, subject to ||x||=1.

[0126] SVD decomposition is performed on A to obtain the following formula:

[0127] A=UDV T .

[0128] Due to the invariance of the orthogonal matrix, the orthogonal matrix U can be directly removed, and further derivation is as follows:

[0129] argmin||Ax||=argmin||UDV T x||=argmin||DV T x||.

[0130] Let y=V T x, and the optimization problem is equivalent to:

[0131] argmin||Dy||.

[0132] The matrix D is a diagonal matrix composed of the eigenvalues of the SVD decomposition of the matrix A, and the eigenvalues of the diagonal matrix are arranged in descending order, so when y=[0 0 … 1] T , argmin||Dy|| is minimum.

[0133] Since x=Vy, and V is a matrix composed of the eigenvectors of the matrix A, therefore, x is the last column of the characteristic matrix V, and thus the homographic transformation matrix M h .

[0134] In an exemplary embodiment, the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected into the calibration plane according to the following formula:

[0135]

[0136] where (x mi ,y mi ,zmi ) is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, (x si ,y si ) is the two-dimensional coordinate of the i-th calibration point on the calibration plane, k i is the depth value of the i-th calibration point on the calibration plane, k i is theoretically 0.

[0137] The homographic change matrix is calculated according to the following formula:

[0138]

[0139] Wherein, α is a scale transformation factor, M h is the homographic change matrix, (x oi ,y oi ) is the two-dimensional coordinate of the i-th calibration point on the ultrasound image.

[0140] A5, store the transformation matrix and the homographic change matrix, complete the calibration of the ultrasound probe; according to the transformation matrix and the homographic change matrix, the two-dimensional coordinate of any target point on the ultrasound image can be converted into the three-dimensional coordinate of the target point in the ultrasound probe coordinate system.

[0141] After completing the above steps of ultrasound probe calibration, any two-dimensional point coordinate on the ultrasound image can be converted into the three-dimensional coordinate in the ultrasound probe coordinate system in the subsequent process.

[0142] Let the coordinate of any two-dimensional point on the ultrasound image be o' i =[x' oi y' oi ] T , let the point coordinate in the calibration plane coordinate system be s' i =[x' si y' si 0] T , according to the homographic matrix change principle, there is the following relationship:

[0143]

[0144] Then, using the transformation matrix M p of the ultrasound probe coordinate system to the calibration plane coordinate system, let the three-dimensional point coordinate m' i =[x' mi y' mi z' mi ] T in the ultrasound probe coordinate system, then there is the following relationship:

[0145]

[0146] In one exemplary embodiment, after the transformation matrix and the homographic transformation matrix are obtained through the above scheme, the two are stored, and in actual application, the two-dimensional coordinates of any target point on the ultrasound image can be converted into the three-dimensional coordinates of the target point in the ultrasound probe coordinate system according to the transformation matrix and the homographic transformation matrix, specifically including the following steps:

[0147] The two-dimensional coordinates of any target point on the ultrasound image are obtained, and the two-dimensional coordinates of the target point on the calibration plane are calculated according to the two-dimensional coordinates of the target point on the ultrasound image and the homographic transformation matrix.

[0148] The three-dimensional coordinates of the target point in the ultrasound probe coordinate system are calculated according to the two-dimensional coordinates of the target point on the calibration plane and the transformation matrix.

[0149] Based on the same inventive concept, the embodiments of the present application also provide a device for implementing the ultrasound probe calibration method based on data dimension reduction and homographic transformation. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more data dimension reduction and homographic transformation based ultrasound probe calibration device embodiments provided below can be referred to the limitations of the data dimension reduction and homographic transformation based ultrasound probe calibration method in the above, which will not be repeated here.

[0150] In one exemplary embodiment, as shown in Figure 4 a data dimension reduction and homographic transformation based ultrasound probe calibration device is provided, comprising:

[0151] A binocular camera, an ultrasound probe, an N-line calibration tool and a data processing device; visual markers are installed on the ultrasound probe and the N-line calibration tool, and the poses of the ultrasound probe and the N-line calibration tool can be tracked and obtained by using the binocular camera; the N-line calibration tool is a three-layer N-line calibration tool based on binary coding.

[0152] The data processing device is used for: selecting a plurality of calibration points on an ultrasound image acquired by an ultrasound probe, and calculating two-dimensional coordinates of each calibration point on the ultrasound image based on an N-line calibration tool; converting the two-dimensional coordinates of each calibration point to a three-dimensional coordinate of each calibration point in an ultrasound probe coordinate system based on a pose relationship among the binocular camera, the ultrasound probe and the N-line calibration tool; determining a calibration plane coordinate system based on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system, and determining a transformation matrix based on the calibration plane coordinate system and the ultrasound probe coordinate system; projecting the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system into the calibration plane according to the transformation matrix, and calculating a homographic transformation matrix according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasound image; storing the transformation matrix and the homographic transformation matrix, and completing calibration of the ultrasound probe; and converting a two-dimensional coordinate of any target point on the ultrasound image into a three-dimensional coordinate of the target point in the ultrasound probe coordinate system according to the transformation matrix and the homographic transformation matrix.

[0153] Of course, Figure 4 The illustrated architecture is only exemplary, and in the implementation of different functions, one or at least two components in the system can be omitted Figure 4 in the system shown.

[0154] In one specific embodiment, to verify the ultrasound probe calibration method and device provided in the above embodiments of the application, an ultrasound probe calibration scene is built as shown in Figure 5 As shown, it includes a set of binocular camera system (DHST-1, Beijing Dahua Wangda Technology Co., Ltd.), single camera resolution 1600*1200, frame rate ≥39fps, positioning error less than 0.3mm; the ultrasound equipment adopts DW360 produced by Dawei company, the probe selects line array probe, and the detection depth is set to 9cm. The notebook used for processing data is DELL G77590, the processor is Inter(R)Core(TM)i7-8750H, the graphics card model is NVIDIA Geforce RTX2060, and the memory is 16GB. An image acquisition card (UB530, TCHD Digital Video Technology Development (Beijing) Co., Ltd.) is used to acquire the desktop image of the ultrasound equipment and transmit it to the workstation in real time, so as to facilitate real-time processing.

[0155] The 3D printing of the calibration tool is performed using photosensitive resin material. The black part of the visual marker is assembled after printing with black nylon. The thin line of the calibration line is printed with a 0.2 mm diameter nylon line. The thick line part is sleeved with a 0.8 mm thick sleeve outside the thin line. In order to ensure the installation accuracy of the nylon line, a dumbbell-shaped groove is arranged on the installation column, as shown in Figure 6 After installation, the nylon line is pulled tight, and the two ends are fixed with screws.

[0156] The coordinates {x, y, z} of the line segment vertices in the calibration coordinate system are measured in Solidworks 2020, as shown in Table 1:

[0157] Table 1 Coordinates of line segment vertices in calibration tool coordinate system

[0158]

[0159] After copying the ultrasound image by image acquisition, the interactive point selection is performed on the software built based on Visual studio 2019, QT 5.12.9, OpenGL4.5 and OpenCV3.3, as shown in Figures 7-9 First, select the point to be picked up in the interface Figure 7 , and then click the center position of the point projection on the image to complete the selection, until all the points are selected. As shown in Figure 8 , the first and second rows of calibration points are complete, and all 8 points can be identified, so the point selection is relatively easy. In Figure 9 , there are only 7 points in the first row and only 6 points in the second row. In this case, the size of the projection must be used for identification to prevent point selection errors.

[0160] After completing the point selection, all image calibration points are combined, and M p and M h are calculated by the method provided in the above embodiments, which are saved as the calibration results.

[0161] Next, in this embodiment, Plus Toolkit is used for calibration accuracy comparison. Plus Toolkit is an open source toolkit for data acquisition, preprocessing and calibration of ultrasound image guided intervention functions. It has been continuously updated and iterated since 2011 and has become one of the most authoritative open source libraries in the field of ultrasound image guidance. The function to be used is vtkPlus Probe CalibrationAlgo::Compute Image To Probe TransformBy LinearLeastSquares Method, and its calibration tool is as follows:

[0162]

[0163] In this embodiment, the error is calculated in the following way: two methods are used respectively to calculate the two-dimensional calibration point projection to the three-dimensional space coordinates m' i =[x' mi y' mi z' mi ] T , and the point coordinates m i =[x mi y mi z mi ] T are calculated by using the N-line model directly. Comparison is made, and the average distance is calculated as the calibration error, which is denoted as e as follows:

[0164]

[0165] A total of three sets of calibration experiments are performed, and 3 groups, 9 groups and 14 groups of data are collected, which include ultrasonic images, poses of the calibration tool under the camera ultrasonic probe pose The three sets of calibration errors calculated are shown in Table 2.

[0166] Table 2. Calibration errors of three sets of experiments

[0167] First group Second group Third group Number of pictures 3 9 14 PLUS error 1.03 ± 0.88 mm 1.10 ± 0.82 mm 1.05 ± 0.63 mm DRHT error 1.02 ± 0.88 mm 1.10 ± 0.79 mm 1.04 ± 0.63 mm

[0168] As can be seen from the three sets of data, the error of the DRHT algorithm is 1.05 mm, which is better than the 1.06 mm of the PLUS algorithm, and also has an advantage in variance. Therefore, for the same data, the DRHT is superior to the PLUS library algorithm in terms of accuracy. It is worth mentioning that the DRHT does not use any nonlinear optimization when calculating, and if the LM algorithm is used for optimization, the accuracy is expected to be further improved.

[0169] From another angle, the calibration errors of the three sets of experiments are not much different, and the calibration error of using 3 groups of data is almost the same as that of using 14 groups of data. In order to further explore the robustness of the DRHT and how many pictures are needed to complete the calibration, a second experiment is designed in this embodiment, 20 groups of data are collected, and 17 experiments are performed, 1-17 groups of data are used for calibration, and 3 groups of data are used for verification, and the verification data do not participate in the calibration. In order to ensure the credibility of the experiment, the calibration data and the verification data are randomly selected, and the specific implementation way is that 20 groups of data are reordered each time, the first n groups of data are used for calibration, and the last 3 groups of data are used for verification. Finally, four indicators are measured, which are the calibration error of the DRHT, the verification error, and the calibration error and the verification error of the PLUS library.

[0170] The calibration control experiment result curve is drawn after statistics, as shown in Figure 10 It can be seen from the figure that when there is only one calibration picture, the calibration error is less than 0.5 mm, but the verification error is close to 1.3, indicating that the calibration tool is over-fitted at this time. After more than two pictures are used, the calibration error rises and eventually tends to be stable, the average DRHT calibration error is 0.9483 mm, and the average PLUS calibration error is 0.9524 mm. The verification error is stable below 1.16 mm, the DRHT verification error is 1.0242 mm, and the PLUS verification error is 1.0276 mm. Whether in terms of calibration accuracy or verification accuracy, DRHT is not inferior.

[0171] Due to the randomness of calibration and data, the verification error has a large fluctuation, such as when the verification error drops to below 0.8 mm in the 6th and 12th groups. However, it can be seen that as the calibration data increases, the fluctuation of the calibration error becomes smaller and smaller, and eventually tends to be stable.

[0172] It can be seen from the first experiment that the DRHT model proposed in the embodiment has feasible verification accuracy, and in the case of not using a nonlinear optimization algorithm, the result is still better than the calibration algorithm of the PLUS library. The second experiment uses the cross-validation method to calculate the calibration error and verification error of the two algorithms, and from the results it can be seen that by using the three-layer 24-line calibration tool proposed in the embodiment, only 2 pictures are needed to provide sufficient calibration information, so as to complete the calibration. Compared with the method of needing 100s to calibrate

[20] or the method of needing thousands of pictures to calibrate, it has greater advantages.

[0173] In an exemplary embodiment, a computer device is provided, which can be a server or a terminal, and its internal structure diagram can be as shown in Figure 11 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement an ultrasonic probe calibration method based on data dimension reduction and homographic transformation.

[0174] Those skilled in the art can understand that Figure 11 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0175] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method embodiments when executing the computer program.

[0176] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0177] In an exemplary embodiment, a computer program product is provided, including a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0178] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0179] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0180] The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0181] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0182] The principles and implementation modes of the present application are described by applying specific examples in the present application. The above-mentioned embodiments are only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A method for ultrasound probe calibration based on data dimension reduction and homographic transformation, characterized in that, The application relates to a method for calibrating an ultrasonic probe. The method comprises the following steps: selecting a plurality of calibration points on an ultrasonic image acquired by the ultrasonic probe, and calculating two-dimensional coordinates of the calibration points on the ultrasonic image based on an N-line calibration tool; the N-line calibration tool is a three-layer N-line calibration tool based on binary encoding; The two-dimensional coordinates of the calibration points are converted to the ultrasonic probe coordinate system based on the pose relationship among a binocular camera, the ultrasonic probe and the N-line calibration tool, and three-dimensional coordinates of the calibration points in the ultrasonic probe coordinate system are obtained; the ultrasonic probe and the N-line calibration tool are both provided with visual markers, and the pose of the ultrasonic probe and the N-line calibration tool can be tracked and acquired by the binocular camera; A calibration plane coordinate system is determined based on the three-dimensional coordinates of the calibration points in the ultrasonic probe coordinate system, and a transformation matrix is determined based on the calibration plane coordinate system and the ultrasonic probe coordinate system; The three-dimensional coordinates of the calibration points in the ultrasonic probe coordinate system are projected into the calibration plane according to the transformation matrix, and a homographic transformation matrix is calculated according to the two-dimensional coordinates of the calibration points on the calibration plane and the two-dimensional coordinates of the calibration points on the ultrasonic image; The transformation matrix and the homographic transformation matrix are stored, and the calibration of the ultrasonic probe is completed; according to the transformation matrix and the homographic transformation matrix, the two-dimensional coordinates of any target point on the ultrasonic image can be converted into the three-dimensional coordinates of the target point in the ultrasonic probe coordinate system.

2. The data dimensionality reduction and homographic transform based ultrasound probe calibration method of claim 1, wherein, The two-dimensional coordinates of any calibration point on the ultrasonic image are calculated according to the following formula: wherein q i is the two-dimensional coordinate of the i-th calibration point on the ultrasound image, q i-1 and q i+1 are the two-dimensional coordinates of the two calibration points adjacent to the i-th calibration point on the ultrasound image; p i and p i+1 are the two-dimensional coordinates of the i-th end point and the i+1-th end point of the N-wire calibration tool on the ultrasound image, respectively.

3. The data dimensionality reduction and homographic transform based ultrasound probe calibration method of claim 2, wherein, The two-dimensional coordinates of the calibration points are converted to the ultrasonic probe coordinate system according to the following formula: wherein q i is the two-dimensional coordinate of the i-th calibration point on the ultrasound image, m i is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, is the relative pose of the ultrasound probe coordinate system in the binocular camera coordinate system, is the relative pose of the N-wire calibration tool in the binocular camera coordinate system.

4. The data dimensionality reduction and homographic transform based ultrasound probe calibration method of claim 1, wherein, A calibration plane coordinate system is determined based on the three-dimensional coordinates of the calibration points in the ultrasonic probe coordinate system, and a transformation matrix is determined based on the calibration plane coordinate system and the ultrasonic probe coordinate system, specifically comprising: A least square method is used to perform plane fitting on the three-dimensional coordinates of the calibration points in the ultrasonic probe coordinate system, and a calibration plane equation is obtained; The calibration plane equation is converted into a least square matrix, and the three-dimensional coordinates of the calibration points in the ultrasonic probe coordinate system are substituted into the least square matrix, so that a normal vector of the calibration plane is solved; Any two points in the calibration plane are selected to form a plane vector, and a calibration plane coordinate system is constructed based on the plane vector and the normal vector of the calibration plane; The transformation matrix is determined based on the calibration plane coordinate system and the ultrasonic probe coordinate system.

5. The data dimensionality reduction and homographic transform based ultrasound probe calibration method of claim 4, wherein, The calibration plane equation is shown in the following formula: Ax+By+Cz+D=0 (C<>0); Wherein, A, B, C and D are coefficients of the calibration plane equation, and (x, y, z) is the three-dimensional coordinates of any calibration point in the ultrasonic probe coordinate system; The calibration plane equation can be deformed into the following formula: z=ax+by+c; wherein are all intermediate custom coefficients; The least square matrix is shown in the following formula: b=AX; wherein, (x i ,y i ,z i ) are three-dimensional coordinates of the i-th calibration point in the ultrasound probe coordinate system, and i has a value range of [1, n].

6. The data dimensionality reduction and homographic transform based ultrasound probe calibration method of claim 5, wherein, The transformation matrix is shown in the following formula: where M p is the transformation matrix, R is the rotation matrix of the calibration plane coordinate system, t is the translation vector of the origin of the calibration plane coordinate system, t = -Rl1, and l1 is the origin of the calibration plane coordinate system.

7. The data dimensionality reduction and homographic transform based ultrasound probe calibration method of claim 6, wherein, The three-dimensional coordinates of the calibration points in the ultrasonic probe coordinate system are projected into the calibration plane according to the following formula: wherein (x mi ,y mi ,z mi ) is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, (x si ,y si ) is the two-dimensional coordinate of the i-th calibration point on the calibration plane, k i is the depth value of the i-th calibration point on the calibration plane, and k i is 0 in theory.

8. The data dimensionality reduction and homographic transform based ultrasound probe calibration method of claim 7, wherein, The homographic transformation matrix is calculated according to the following formula: where a is a scale transformation factor, M h is a homographic matrix, (x oi ,y oi ) are the two-dimensional coordinates of the i-th calibration point on the ultrasound image.

9. The data dimensionality reduction and homographic transform based ultrasound probe calibration method of claim 1, wherein, According to the transformation matrix and the homographic transformation matrix, the two-dimensional coordinates of any target point on the ultrasonic image are converted into the three-dimensional coordinates of the target point in the ultrasonic probe coordinate system, specifically comprising: Obtaining the two-dimensional coordinates of any target point on the ultrasound image, and calculating the two-dimensional coordinates of the target point on the calibration plane according to the two-dimensional coordinates of the target point on the ultrasound image and the homographic transformation matrix; According to the two-dimensional coordinates of the target point on the calibration plane and the transformation matrix, the three-dimensional coordinates of the target point in the ultrasound probe coordinate system are calculated.

10. An ultrasonic probe calibration device based on data dimension reduction and homographic transformation, characterized in that, Comprise: A binocular camera, an ultrasound probe, an N-line calibration tool and a data processing device; visual markers are installed on the ultrasound probe and the N-line calibration tool, and the poses of the ultrasound probe and the N-line calibration tool can be tracked and obtained by using the binocular camera; the N-line calibration tool is a three-layer N-line calibration tool based on binary encoding; The data processing device is used to select a plurality of calibration points on the ultrasound image obtained by the ultrasound probe, and calculate the two-dimensional coordinates of each calibration point on the ultrasound image based on the N-line calibration tool; based on the pose relationship among the binocular camera, the ultrasound probe and the N-line calibration tool, the two-dimensional coordinates of each calibration point are converted to the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system; based on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system, the calibration plane coordinate system is determined, and based on the calibration plane coordinate system and the ultrasound probe coordinate system, the transformation matrix is determined; according to the transformation matrix, the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected into the calibration plane, and according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasound image, the homographic transformation matrix is calculated; the transformation matrix and the homographic transformation matrix are stored, and the calibration of the ultrasound probe is completed; according to the transformation matrix and the homographic transformation matrix, the two-dimensional coordinates of any target point on the ultrasound image can be converted into the three-dimensional coordinates of the target point in the ultrasound probe coordinate system.

Images