Ultrasonic probe calibration method and device based on data dimension reduction and homography

Abstract

The invention discloses an ultrasonic probe calibration method and device based on data dimension reduction and homography, and relates to the technical field of ultrasonic probe calibration, and the method comprises the steps: firstly, calculating and obtaining two-dimensional coordinates of each calibration point on an ultrasonic image based on an N-line calibration tool, and converting the two-dimensional coordinates into an ultrasonic probe coordinate system; based on the three-dimensional coordinates of the calibration points in the ultrasonic probe coordinate system, a calibration plane coordinate system is determined, and then a transformation matrix between the calibration plane coordinate system and the calibration plane coordinate system is determined; finally, projecting the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system into a calibration plane, and calculating to obtain a homographic change matrix; the calculated transformation matrix and the homography matrix are stored, the two-dimensional coordinate of any target point on the ultrasonic image can be conveniently and accurately converted into the three-dimensional coordinate of the target point in the ultrasonic probe coordinate system subsequently according to the transformation matrix and the homography matrix, and according to the scheme, the existing calibration process can be simplified, and meanwhile the calibration accuracy is improved. And the calibration precision of the ultrasonic probe is improved.

Description

Technical Field

[0001] The present application relates to the technical field of ultrasonic probe calibration, and in particular to an ultrasonic probe calibration method and device based on data dimension reduction and homography transformation. Background Art

[0002] Ultrasonic detection equipment is a common detection equipment in medicine. It is used to see through subcutaneous tissue and has the characteristics of real-time and radiation-free. The ultrasonic probe is the core of the ultrasonic detection equipment. It converts electrical energy into ultrasonic energy through piezoelectric crystals. After contacting the human body, it returns to the ultrasonic vibrator and is converted into an electrical signal, and then converted into a two-dimensional black and white image. Ultrasonic three-dimensional reconstruction is an important way to expand the doctor's field of vision. The distortion of ultrasonic images causes the information provided by a single two-dimensional image to be very limited. The current mainstream research direction is to reconstruct the scanned area in three dimensions through continuous ultrasonic sequences, thereby achieving an effect similar to CT scanning, which is used to guide doctors or even robots to perform surgery.

[0003] Finding the spatial position relationship between ultrasound sequences is the prerequisite for achieving three-dimensional reconstruction. The three mainstream methods are constraint method, image method and tracking method. Among them, tracking method is the most widely used ultrasound three-dimensional reconstruction method, and the additional positioning equipment used includes optical tracking, electromagnetic tracking, accelerometer, etc. For the tracking method, the position calibration between the ultrasound probe and the positioning device (abbreviated as ultrasound probe calibration) is its prerequisite. However, most of the existing ultrasound probe calibration tools have problems such as cumbersome process and low precision.

[0004] Therefore, it is very necessary for the art to provide a new calibration tool for ultrasonic probe calibration method to simplify the existing calibration process and improve the calibration accuracy. Summary of the invention

[0005] The purpose of this application is to provide an ultrasonic probe calibration method and device based on data dimension reduction and homography transformation, which can improve the calibration accuracy of the ultrasonic probe while simplifying the existing calibration process.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides an ultrasonic probe calibration method based on data dimension reduction and homography transformation, comprising the following steps:

[0008] A number of calibration points are selected on the ultrasound image acquired by the ultrasound probe, and the two-dimensional coordinates of each calibration point on the ultrasound image are calculated based on an N-line calibration tool; the N-line calibration tool is a three-layer N-line calibration tool based on binary coding.

[0009] Based on the posture relationship between the binocular camera, the ultrasonic probe and the N-line calibration tooling, the two-dimensional coordinates of each calibration point are converted to the ultrasonic probe coordinate system to obtain the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system; visual markers are installed on the ultrasonic probe and the N-line calibration tooling, and the binocular camera can be used to track and obtain the posture of the ultrasonic probe and the N-line calibration tooling.

[0010] Based on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system, a calibration plane coordinate system is determined, and based on the calibration plane coordinate system and the ultrasound probe coordinate system, a transformation matrix is ​​determined.

[0011] The three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected onto the calibration plane according to the transformation matrix, and the homography change matrix is ​​calculated according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasound image.

[0012] The transformation matrix and the homography change matrix are stored to complete the calibration of the ultrasound probe; based on the transformation matrix and the homography change matrix, the two-dimensional coordinates of any target point on the ultrasound image can be converted into the three-dimensional coordinates of the target point in the ultrasound probe coordinate system.

[0013] Optionally, the two-dimensional coordinates of any calibration point on the ultrasound image are calculated according to the following formula:

[0014]

[0015] Among them, q i is the two-dimensional coordinate of the i-th calibration point on the ultrasound image, q i-1 and q i+1 are the two-dimensional coordinates of the two calibration points adjacent to the i-th calibration point on the ultrasound image; p i and p i+1 The two-dimensional coordinates of the i-th endpoint and the i+1-th endpoint of the N-line calibration tool on the ultrasonic image are respectively obtained.

[0016] Optionally, the two-dimensional coordinates of each calibration point are converted to the ultrasound probe coordinate system according to the following formula:

[0017]

[0018] Among them, q i is the two-dimensional coordinate of the i-th calibration point on the ultrasound image, m i is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, is the relative posture of the ultrasound probe coordinate system in the binocular camera coordinate system, The relative posture of the N-line calibration tool in the binocular camera coordinate system.

[0019] Optionally, based on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system, a calibration plane coordinate system is determined, and based on the calibration plane coordinate system and the ultrasound probe coordinate system, a transformation matrix is ​​determined, specifically including the following steps:

[0020] The least square method is used to perform plane fitting on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system to obtain the calibration plane equation.

[0021] The calibration plane equation is converted into a least squares matrix, and the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are substituted to obtain the normal vector of the calibration plane.

[0022] Two points are randomly selected in the calibration plane to form a plane vector, and a calibration plane coordinate system is constructed based on the plane vector and a normal vector of the calibration plane.

[0023] A transformation matrix is ​​determined based on the calibration plane coordinate system and the ultrasound probe coordinate system.

[0024] Optionally, the calibration plane equation is as shown below:

[0025] Ax+By+Cz+D=0(C≠0).

[0026] Among them, A, B, C, and D are the coefficients of the calibration plane equation, and (x, y, z) is the three-dimensional coordinate of any calibration point in the ultrasonic probe coordinate system.

[0027] The calibration plane equation can be transformed into the following formula:

[0028] z=ax+by+c.

[0029] in, All are intermediate custom coefficients.

[0030] The least squares matrix is ​​shown below:

[0031] b=AX.

[0032] in, (x i ,y i ,z i ) is the three-dimensional coordinate of the i-th calibration point in the ultrasonic probe coordinate system, and the value range of i is [1,n].

[0033] Optionally, the transformation matrix is ​​expressed as follows:

[0034]

[0035] Among them, M pis the transformation matrix, R is the rotation matrix of the calibration plane coordinate system, t is the translation vector of the origin of the calibration plane coordinate system, t=-Rl1, l1 is the origin of the calibration plane coordinate system.

[0036] Optionally, the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected into the calibration plane according to the following formula:

[0037]

[0038] Among them, (x mi ,y mi ,z mi ) is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, (x si ,y si ) is the two-dimensional coordinate of the i-th calibration point on the calibration plane, k i is the depth value of the i-th calibration point on the calibration plane, k i Theoretically, it is 0.

[0039] Optionally, the homography change matrix is ​​calculated according to the following formula:

[0040]

[0041] Among them, α is the scale transformation factor, M h is the homography transformation matrix, (x oi ,y oi ) is the two-dimensional coordinate of the i-th calibration point on the ultrasound image.

[0042] Optionally, according to the transformation matrix and the homography change matrix, converting the two-dimensional coordinates of any target point on the ultrasound image into the three-dimensional coordinates of the target point in the ultrasound probe coordinate system specifically includes:

[0043] The two-dimensional coordinates of any target point on the ultrasound image are obtained, and the two-dimensional coordinates of the target point on the calibration plane are calculated according to the two-dimensional coordinates of the target point on the ultrasound image and the homography change matrix.

[0044] The three-dimensional coordinates of the target point in the ultrasound probe coordinate system are calculated based on the two-dimensional coordinates of the target point on the calibration plane and the transformation matrix.

[0045] In the second aspect, the present application provides an ultrasonic probe calibration device based on data dimensionality reduction and homography transformation, comprising: a binocular camera, an ultrasonic probe, an N-line calibration tool and a data processing device; visual markers are installed on the ultrasonic probe and the N-line calibration tool, and the binocular camera can be used to track and obtain the position and posture of the ultrasonic probe and the N-line calibration tool; the N-line calibration tool is a three-layer N-line calibration tool based on binary coding.

[0046] The data processing device is used to: select a number of calibration points on the ultrasonic image acquired by the ultrasonic probe, and calculate the two-dimensional coordinates of each calibration point on the ultrasonic image based on the N-line calibration tooling; based on the posture relationship between the binocular camera, the ultrasonic probe and the N-line calibration tooling, convert the two-dimensional coordinates of each calibration point to the ultrasonic probe coordinate system to obtain the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system; determine the calibration plane coordinate system based on the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system, and determine the transformation matrix based on the calibration plane coordinate system and the ultrasonic probe coordinate system; project the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system into the calibration plane according to the transformation matrix, and calculate the homography change matrix according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasonic image; store the transformation matrix and the homography change matrix to complete the calibration of the ultrasonic probe; according to the transformation matrix and the homography change matrix, the two-dimensional coordinates of any target point on the ultrasonic image can be converted into the three-dimensional coordinates of the target point in the ultrasonic probe coordinate system.

[0047] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0048] The present application provides an ultrasonic probe calibration method and device based on data dimensionality reduction and homography transformation, the method comprising: selecting a number of calibration points on an ultrasonic image acquired by an ultrasonic probe, and calculating the two-dimensional coordinates of each calibration point on the ultrasonic image based on an N-line calibration tooling; based on the posture relationship between a binocular camera, an ultrasonic probe and an N-line calibration tooling, converting the two-dimensional coordinates of each calibration point to an ultrasonic probe coordinate system, and then determining a calibration plane coordinate system based on the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system, and determining a transformation matrix based on the calibration plane coordinate system and the ultrasonic probe coordinate system; then projecting the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system onto the calibration plane according to the transformation matrix, and calculating a homography change matrix; storing the above-calculated transformation matrix and homography change matrix to complete the calibration of the ultrasonic probe; and subsequently, according to the transformation matrix and the homography change matrix, conveniently and accurately converting the two-dimensional coordinates of any target point on the ultrasonic image into the three-dimensional coordinates of the target point in the ultrasonic probe coordinate system. The solution provided in the present application fits the ultrasound imaging plane, uses the plane to reduce the three-dimensional points into two-dimensional points, and then uses homography changes to characterize the relationship between each two-dimensional point after dimensionality reduction and each point on the ultrasound image. This eliminates the scale factor used in the current mainstream calibration tooling, avoids the problem of calibration error amplification caused by scale factor estimation errors, and can improve the calibration accuracy of the ultrasound probe while simplifying the existing calibration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flowchart of an ultrasonic probe calibration method based on data dimension reduction and homography transformation is provided in one embodiment of the present application.

[0051] Figure 2 A schematic diagram of an N-line calibration tooling in an ultrasonic probe calibration method based on data dimensionality reduction and homography transformation provided in one embodiment of the present application.

[0052] Figure 3 This is a flowchart of step A3 in a method for calibrating an ultrasonic probe based on data dimension reduction and homography transformation provided in one embodiment of the present application.

[0053] Figure 4 A schematic diagram of functional modules of an ultrasonic probe calibration device based on data dimension reduction and homography transformation provided in one embodiment of the present application.

[0054] Figure 5 A schematic diagram of an ultrasound probe calibration scene constructed in an ultrasound probe calibration method based on data dimensionality reduction and homography transformation provided in another embodiment of the present application.

[0055] Figure 6 A schematic diagram of a calibration tool in an ultrasonic probe calibration method based on data dimensionality reduction and homography transformation provided in another embodiment of the present application.

[0056] Figure 7 A schematic diagram of an interface for picking calibration points in an ultrasonic probe calibration method based on data dimensionality reduction and homography transformation provided in another embodiment of the present application.

[0057] Figure 8 A schematic diagram of the result of completely picking up calibration points in an ultrasonic probe calibration method based on data dimensionality reduction and homography transformation provided in another embodiment of the present application.

[0058] Fig. 9 A schematic diagram of a result in which calibration points cannot be completely picked up in an ultrasonic probe calibration method based on data dimensionality reduction and homography transformation provided in another embodiment of the present application.

[0059] Fig.10 A schematic diagram of a set of calibration control experiment result curves in an ultrasonic probe calibration method based on data dimensionality reduction and homography transformation provided in another embodiment of the present application.

[0060] Fig.11 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0063] In an exemplary embodiment, Figure 1 The flowchart shown provides an ultrasonic probe calibration method based on data dimension reduction and homography transformation, comprising the following steps:

[0064] A1. Select several calibration points on the ultrasound image acquired by the ultrasound probe, and calculate the two-dimensional coordinates of each calibration point on the ultrasound image based on the N-line calibration tool; the N-line calibration tool is a three-layer N-line calibration tool based on binary coding.

[0065] The traditional N-line model has only three line segments, such as Figure 2 middle Assuming that there are intersection points q0, q1, and q2 under the ultrasound image, only point q1 can directly calculate the coordinates through the N-line model principle, so the utilization rate of the calibration points is only 1 / 3.

[0066] In order to increase the number and utilization of calibration points, improve the calibration accuracy and reduce the requirement for the number of calibration images. In this embodiment, the N-line model is expanded and a three-layer 24-line model is designed. A calibration plane contains 8 calibration line segments (solid lines). The dotted line is the intersection of the ultrasound imaging plane and the calibration plane, and the intersection points are q0~q 23 In this way, when all calibration points are identified, q1~q6, q9~q 14 ,q 17 ~q 22 The coordinates of a total of 18 points can be directly calculated through the N-line model, and the utilization rate of the calibration points reaches 3 / 4.

[0067] According to the principle of similar triangles, remove the points that cannot be calculated at both ends. 14 ,q 17 ~q 22 For , the following relationship holds:

[0068]

[0069] in, It is a proportional relationship in three-dimensional space and can be represented by the pixel distance on the ultrasound image:

[0070]

[0071] According to the vector operation rules:

[0072]

[0073] In the actual model making process, the calibration line segments are also coded by the thickness of the line for easy identification. The thin line represents 0 and the thick line represents 1. Starting from the left column, it is {000, 001, 010, ..., 111}, which represents the numbers 0 to 7 when converted to decimal. This is why each calibration plane uses 8 line segments.

[0074] In an exemplary embodiment, in step A1, the two-dimensional coordinates of any calibration point on the ultrasound image are calculated according to the following formula:

[0075]

[0076] Among them, q i is the two-dimensional coordinate of the i-th calibration point on the ultrasound image, q i-1 and q i+1 are the two-dimensional coordinates of the two calibration points adjacent to the i-th calibration point on the ultrasound image; p i and p i+1 The two-dimensional coordinates of the i-th endpoint and the i+1-th endpoint of the N-line calibration tool on the ultrasonic image are respectively obtained.

[0077] A2. Based on the posture relationship between the binocular camera, the ultrasonic probe and the N-line calibration tooling, the two-dimensional coordinates of each calibration point are converted to the ultrasonic probe coordinate system to obtain the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system; visual markers are installed on the ultrasonic probe and the N-line calibration tooling, and the binocular camera can be used to track and obtain the posture of the ultrasonic probe and the N-line calibration tooling.

[0078] Assume that the N-line calibration tool posture of the calibration tool under the binocular camera is The ultrasound probe posture under the camera is Assume that the coordinates of the calibration point in the ultrasonic probe coordinate system are m i =[x mi ,y mi ,z mi ] T, based on the relationship of the spatial coordinate system, in this embodiment, the two-dimensional coordinates of each calibration point can be converted to the ultrasound probe coordinate system according to the following formula:

[0079]

[0080] Among them, q i is the two-dimensional coordinate of the i-th calibration point on the ultrasound image, m i is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, is the relative posture of the ultrasound probe coordinate system in the binocular camera coordinate system, The relative posture of the N-line calibration tool in the binocular camera coordinate system.

[0081] A3. Based on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system, determine the calibration plane coordinate system, and based on the calibration plane coordinate system and the ultrasound probe coordinate system, determine the transformation matrix.

[0082] Since ultrasound imaging is a plane, all points are theoretically in the same plane in the ultrasound probe coordinate system. Therefore, the plane-plane homography transformation matrix can be used to represent the mapping relationship between the two-dimensional coordinates of the image and the three-dimensional coordinates in the ultrasound probe coordinate system. However, the three-dimensional coordinates in the ultrasound probe coordinate system need to be converted into two-dimensional coordinates first. Therefore, consider reducing the dimension by projecting the points onto the plane in the ultrasound probe coordinate system.

[0083] The least squares method is used to perform plane fitting on the three-dimensional coordinates in the ultrasound probe coordinate system. The general expression of the plane equation is:

[0084] Ax+By+Cz+D=0(C≠0).

[0085] Transformed into:

[0086]

[0087] make Right now:

[0088] z=ax+by+c.

[0089] This can be transformed into a least squares matrix b = AX, where

[0090] Substituting the three-dimensional coordinates of the point in the ultrasound probe coordinate system, a, b, and c can be calculated, and the normal vector of the calibration plane can be obtained.

[0091] In an exemplary embodiment, Figure 3 In the flowchart shown, step A3 specifically includes the following steps:

[0092] A31. Use the least square method to perform plane fitting on the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system to obtain the calibration plane equation. The calibration plane equation is shown in the following formula:

[0093] Ax+By+Cz+D=0(C≠0).

[0094] Among them, A, B, C, and D are the coefficients of the calibration plane equation, and (x, y, z) is the three-dimensional coordinate of any calibration point in the ultrasonic probe coordinate system.

[0095] The calibration plane equation can be transformed into the following formula:

[0096] z=ax+by+c.

[0097] in, All are intermediate custom coefficients.

[0098] A32. Convert the calibration plane equation into a least squares matrix, and substitute the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system to obtain the normal vector of the calibration plane. In this embodiment, the least squares matrix is ​​shown as follows:

[0099] b=AX.

[0100] in, (x i ,y i ,z i ) is the three-dimensional coordinate of the i-th calibration point in the ultrasonic probe coordinate system, and the value range of i is [1,n].

[0101] A33. Select any two points in the calibration plane to form a plane vector, and construct the calibration plane coordinate system based on the plane vector and the normal vector of the calibration plane.

[0102] Next, establish a coordinate system in the plane and substitute any two points l1 and l2 as shown below:

[0103]

[0104] make From the vector product formula Can be found Then the rotation matrix of the new coordinate system is as follows:

[0105]

[0106] Then arbitrarily specify a point as the origin, assuming it is l1, then the translation vector is as shown below:

[0107] t=-Rl1.

[0108] S34, based on the calibration plane coordinate system and the ultrasound probe coordinate system, determine the transformation matrix. The transformation matrix is ​​expressed as follows:

[0109]

[0110] Among them, M p is the transformation matrix, R is the rotation matrix of the calibration plane coordinate system, t is the translation vector of the origin of the calibration plane coordinate system, t=-Rl1, l1 is the origin of the calibration plane coordinate system.

[0111] A4. The three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected onto the calibration plane according to the transformation matrix, and the homography change matrix is ​​calculated according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasound image.

[0112] Through the transformation matrix M p All three-dimensional points can be projected into the two-dimensional calibration plane, and the point s in the calibration plane is i ={x si ,y si} T , that is, the following relationship:

[0113]

[0114] Among them, k i Theoretically, the value should be zero, but due to the existence of errors, it is often not zero and is directly discarded.

[0115] At this point, the projection of the three-dimensional coordinates to the two-dimensional calibration plane is completed. Next, the homography transformation matrix M from the two-dimensional point set {o} to {s} needs to be calculated. h , assuming the following relationship exists:

[0116]

[0117] Among them, α is the scale transformation factor. After expanding and eliminating α, we get the following formula:

[0118]

[0119] After deformation, the following formula can be obtained:

[0120]

[0121] It can be further transformed into the least squares matrix Ax=0, where:

[0122]

[0123] x=[h 11 h 12 h 13h 21 h 22 h 23 h 31 h 32 h 33 ] T .

[0124] Ax=0 is an overdetermined equation with no exact solution, so it is estimated through SVD decomposition. Assuming that there is a set of x that makes Ax=0, then x multiplied by any proportional coefficient k obviously still holds true. Therefore, we might as well add the constraint ||x||=1, thus constructing a constrained optimization problem, as shown in the following formula:

[0125] argmin||Ax||,subject to||x||=1.

[0126] Perform SVD decomposition on A and get the following formula:

[0127] A=UDV T .

[0128] Due to the norm preservation of orthogonal matrices, the orthogonal matrix U can be directly removed, and further deduced as follows:

[0129] argmin||Ax||=argmin||UDV T x||=argmin||DV T x||.

[0130] Let y = V T x, then the optimization problem is equivalent to:

[0131] argmin||Dy||.

[0132] The matrix D is a diagonal matrix composed of the eigenvalues ​​of the SVD decomposition of the matrix A, and the eigenvalues ​​of the diagonal matrix are arranged in descending order, so when y = [0 0 … 1] T When argmin||Dy|| is minimum.

[0133] Since x = Vy, and V is the matrix composed of the eigenvectors of matrix A, x is the last column of the characteristic matrix V. So far, the homography change matrix M can be obtained. h .

[0134] In an exemplary embodiment, the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected into the calibration plane according to the following formula:

[0135]

[0136] Among them, (x mi ,y mi ,zmi ) is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, (x si ,y si ) is the two-dimensional coordinate of the i-th calibration point on the calibration plane, k i is the depth value of the i-th calibration point on the calibration plane, k i Theoretically, it is 0.

[0137] The homography change matrix is ​​calculated according to the following formula:

[0138]

[0139] Among them, α is the scale transformation factor, M h is the homography transformation matrix, (x oi ,y oi ) is the two-dimensional coordinate of the i-th calibration point on the ultrasound image.

[0140] A5. Store the transformation matrix and homography change matrix to complete the calibration of the ultrasound probe. According to the transformation matrix and homography change matrix, the two-dimensional coordinates of any target point on the ultrasound image can be converted into the three-dimensional coordinates of the target point in the ultrasound probe coordinate system.

[0141] After completing the ultrasonic probe calibration in the above steps, in the subsequent process, the coordinates of any two-dimensional point on the ultrasonic image can be converted into three-dimensional coordinates in the ultrasonic probe coordinate system.

[0142] Suppose the coordinates of any two-dimensional point on the ultrasound image are o' i =[x' oi y' oi ] T , let the coordinates of the point in the calibration plane coordinate system be s' i =[x' si y' si 0] T , according to the principle of homography matrix change, there is the following relationship:

[0143]

[0144] Then use the transformation matrix M from the ultrasound probe coordinate system to the calibration plane coordinate system p , let the three-dimensional point coordinate m' in the ultrasound probe coordinate system i =[x' mi y' mi z' mi ] T , then there is the following relationship:

[0145]

[0146] In an exemplary embodiment, after obtaining the transformation matrix and the homography change matrix through the above scheme, the two are stored. In actual application, the two-dimensional coordinates of any target point on the ultrasound image can be converted into the three-dimensional coordinates of the target point in the ultrasound probe coordinate system according to the transformation matrix and the homography change matrix, which specifically includes the following steps:

[0147] The two-dimensional coordinates of any target point on the ultrasound image are obtained, and the two-dimensional coordinates of the target point on the calibration plane are calculated according to the two-dimensional coordinates of the target point on the ultrasound image and the homography change matrix.

[0148] The three-dimensional coordinates of the target point in the ultrasound probe coordinate system are calculated based on the two-dimensional coordinates of the target point on the calibration plane and the transformation matrix.

[0149] Based on the same inventive concept, the embodiment of the present application also provides a device for implementing the ultrasonic probe calibration method based on data dimensionality reduction and homography transformation involved in the above embodiment. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the ultrasonic probe calibration device based on data dimensionality reduction and homography transformation provided below can refer to the limitations of the ultrasonic probe calibration method based on data dimensionality reduction and homography transformation in the above text, and will not be repeated here.

[0150] In an exemplary embodiment, Figure 4 As shown, an ultrasonic probe calibration device based on data dimension reduction and homography transformation is provided, comprising:

[0151] Binocular camera, ultrasonic probe, N-line calibration tool and data processing equipment; visual markers are installed on the ultrasonic probe and the N-line calibration tool, and the binocular camera can be used to track and obtain the position and posture of the ultrasonic probe and the N-line calibration tool; the N-line calibration tool is a three-layer N-line calibration tool based on binary coding.

[0152] The data processing device is used to: select a number of calibration points on the ultrasonic image acquired by the ultrasonic probe, and calculate the two-dimensional coordinates of each calibration point on the ultrasonic image based on the N-line calibration tooling; based on the posture relationship between the binocular camera, the ultrasonic probe and the N-line calibration tooling, convert the two-dimensional coordinates of each calibration point to the ultrasonic probe coordinate system to obtain the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system; determine the calibration plane coordinate system based on the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system, and determine the transformation matrix based on the calibration plane coordinate system and the ultrasonic probe coordinate system; project the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system into the calibration plane according to the transformation matrix, and calculate the homography change matrix according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasonic image; store the transformation matrix and the homography change matrix to complete the calibration of the ultrasonic probe; according to the transformation matrix and the homography change matrix, the two-dimensional coordinates of any target point on the ultrasonic image can be converted into the three-dimensional coordinates of the target point in the ultrasonic probe coordinate system.

[0153] certainly, Figure 4 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different functions. Figure 4 One or at least two components of the system shown.

[0154] In a specific embodiment, in order to verify the ultrasound probe calibration method and device provided in the above embodiment of the present application, an ultrasound probe calibration scene is constructed as follows: Figure 5 As shown in the figure, it includes a binocular camera system (DHST-1, Beijing Dahua Wangda Technology Co., LTD), a single camera resolution of 1600*1200, frame rate ≥39fps, positioning error less than 0.3mm; the ultrasound equipment uses DW360 produced by Dawei Company, the probe uses a linear array probe, and the detection depth is set to 9cm. The notebook used to process the data is DELL G77590, the processor is Inter(R) Core(TM) i7-8750H, the graphics card model is NVIDIA Geforce RTX2060, and the memory is 16GB. The image acquisition card (UB530, TCHD Digital Video Technology Development (Beijing) Co, Ltd.) is used to collect the desktop image of the ultrasound equipment and transmit it to the workstation in real time for real-time processing.

[0155] The calibration tooling is 3D printed with photosensitive resin material. The black part of the visual mark is printed with black nylon and then assembled. The thin wire of the calibration line is a 0.2mm diameter nylon wire, and the thick wire is covered with a 0.8mm thick sleeve on the outside of the thin wire. In order to ensure the installation accuracy of the nylon wire, a dumbbell-shaped groove is set on the installation column, such as Figure 6 After installation, tighten the nylon wire and fix the two ends with screws.

[0156] The coordinates {x, y, z} of the line segment vertices measured in Solidworks 2020 in the calibration coordinate system are shown in Table 1:

[0157] Table 1 Coordinates of the vertices of the calibration line segment in the calibration tooling coordinate system

[0158]

[0159] After copying the ultrasound image using image acquisition, interactive point selection was performed on the software built on Visual studio 2019, QT 5.12.9, OpenGL4.5 and OpenCV3.3, such as Figure 7 to Figure 9 As shown. First, Figure 7 Select the point to be picked up in the interface, and then click the center of the projection of the point on the image to complete the selection until all points are selected. Figure 8 In , the first and second rows of calibration points are complete, and all 8 points can be identified. In this case, point selection is relatively easy. Fig. 9 In the figure, there are only 7 points in the first row and 6 points in the second row. In this case, it is necessary to use the size of the projection to identify them to prevent incorrect point selection.

[0160] After the point selection is completed, all the image calibration points are merged, and M is calculated by the method provided in the above embodiment of the present application. p and M h , and save it as the calibration result.

[0161] Next, in this embodiment, Plus Toolkit is used to compare the calibration accuracy. Plus Toolkit is an open source toolkit for data acquisition, preprocessing and calibration of ultrasound image-guided intervention. It has been continuously updated and iterated since 2011 and has become one of the most authoritative open source libraries in ultrasound image guidance. The function to be used is vtkPlus Probe CalibrationAlgo::Compute Image To Probe TransformBy LinearLe-astSquares Method, and its calibration tool is shown as follows:

[0162]

[0163] In this embodiment, the error is calculated by using two methods to calculate the projection of the two-dimensional calibration point to the three-dimensional space coordinate m' i =[x' mi y' mi z' mi ] T , and the point coordinates m calculated directly using the N-line model i =[x mi y mi z mi ] T For comparison, the average distance is calculated as the calibration error, which is set as e as follows:

[0164]

[0165] A total of three calibration experiments were conducted, collecting 3, 9 and 14 sets of data respectively. The data included ultrasound images and the posture of the calibration tool under the camera. Ultrasound probe posture The three groups of calibration errors are calculated and shown in Table 2.

[0166] Table 2 Calibration errors of three groups of experiments

[0167] Group 1 Group 2 Group 3 Number of pictures 3 9 14 PLUS Error 1.03±0.88mm 1.10±0.82mm 1.05±0.63mm DRHT Error 1.02±0.88mm 1.10±0.79mm 1.04±0.63mm

[0168] From the three sets of data, we can see that the error of the DRHT algorithm is 1.05mm, which is better than the 1.06mm of the PLUS algorithm, and it also has an advantage in variance. Therefore, for the same data, DRHT is better than the algorithm of the PLUS library in terms of accuracy. It is worth mentioning that no nonlinear optimization is used in the DRHT calculation. If algorithms such as LM are used for optimization, the accuracy is expected to be further improved.

[0169] From another perspective, the calibration errors of the three groups of experiments are not much different. The calibration error using 3 groups of data is almost the same as the calibration error using 14 groups of data. In order to further explore the robustness of DRHT and how many pictures are needed to complete the calibration, this embodiment designs a second experiment, collects 20 groups of data, conducts 17 experiments, uses 1 to 17 groups of data for calibration, and 3 groups of data for verification. The verification data does not participate in the calibration. In order to ensure the credibility of the experiment, the calibration data and verification data are randomly selected. The specific implementation method is to re-sort the 20 groups of data for each experiment. The nth group of experiments uses the first n groups of data for calibration, and the last 3 groups of data for verification. Finally, four indicators are measured, namely the calibration error and verification error of DRHT and the calibration error and verification error of the PLUS library.

[0170] After statistics, the calibration control experiment result curve is drawn as follows Fig.10 As shown. As can be seen from the figure, when there is only one calibration picture, the calibration error is less than 0.5mm, but the verification error is close to 1.3, which means that the calibration tooling is a bit overfitted at this time. After using more than two pictures, the calibration error increases and eventually stabilizes. The average DRHT calibration error is 0.9483mm, and the average PLUS calibration error is 0.9524mm. The verification error is stable below 1.16mm, the DRHT verification error is 1.0242mm, and the PLUS verification error is 1.0276mm. Whether in terms of calibration accuracy or verification accuracy, DRHT is not inferior.

[0171] Due to the randomness of calibration and data, the verification error has large fluctuations, such as the verification error dropped below 0.8 mm in groups 6 and 12. However, it can be seen that with the increase of calibration data, the fluctuation of calibration error becomes smaller and smaller, and finally tends to be stable.

[0172] From the first experiment, it can be seen that the verification accuracy of the DRHT model proposed in this embodiment is feasible. In the absence of a nonlinear optimization algorithm, the result is still better than the calibration algorithm of the PLUS library. The second experiment uses a cross-validation method to calculate the calibration error and verification error of the two algorithms. From the results, it can be seen that the three-layer 24-line calibration tool proposed in this embodiment only requires two pictures to provide sufficient calibration information to complete the calibration, which is more advantageous than the method that requires 100 seconds for calibration

[20] or the method that requires thousands of pictures for calibration.

[0173] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an ultrasonic probe calibration method based on data dimensionality reduction and homography transformation is implemented.

[0174] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0175] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0176] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0177] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0178] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0179] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0180] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0181] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0182] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An ultrasonic probe calibration method based on data dimension reduction and homography transformation, characterized in that: include: Selecting a number of calibration points on the ultrasound image acquired by the ultrasound probe, and calculating the two-dimensional coordinates of each calibration point on the ultrasound image based on an N-line calibration tool; the N-line calibration tool is a three-layer N-line calibration tool based on binary coding; Based on the posture relationship between the binocular camera, the ultrasonic probe and the N-line calibration tooling, the two-dimensional coordinates of each calibration point are converted to the ultrasonic probe coordinate system to obtain the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system; visual markers are installed on the ultrasonic probe and the N-line calibration tooling, and the binocular camera can be used to track and obtain the posture of the ultrasonic probe and the N-line calibration tooling; Based on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system, a calibration plane coordinate system is determined, and based on the calibration plane coordinate system and the ultrasound probe coordinate system, a transformation matrix is ​​determined; The three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected onto the calibration plane according to the transformation matrix, and a homography change matrix is ​​calculated according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasound image; The transformation matrix and the homography change matrix are stored to complete the calibration of the ultrasound probe; based on the transformation matrix and the homography change matrix, the two-dimensional coordinates of any target point on the ultrasound image can be converted into the three-dimensional coordinates of the target point in the ultrasound probe coordinate system.

2. The ultrasonic probe calibration method based on data dimension reduction and homography transformation according to claim 1 is characterized in that: The two-dimensional coordinates of any calibration point on the ultrasound image are calculated according to the following formula: Among them, q i is the two-dimensional coordinate of the i-th calibration point on the ultrasound image, q i-1 and q i+1 are the two-dimensional coordinates of the two calibration points adjacent to the i-th calibration point on the ultrasound image; p i and p i+1 The two-dimensional coordinates of the i-th endpoint and the i+1-th endpoint of the N-line calibration tool on the ultrasonic image are respectively obtained.

3. The ultrasonic probe calibration method based on data dimension reduction and homography transformation according to claim 2 is characterized in that: The two-dimensional coordinates of each calibration point are converted to the ultrasound probe coordinate system according to the following formula: Among them, q i is the two-dimensional coordinate of the i-th calibration point on the ultrasound image, m i is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, is the relative posture of the ultrasound probe coordinate system in the binocular camera coordinate system, The relative posture of the N-line calibration tool in the binocular camera coordinate system.

4. The ultrasonic probe calibration method based on data dimension reduction and homography transformation according to claim 1, characterized in that: Based on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system, a calibration plane coordinate system is determined, and based on the calibration plane coordinate system and the ultrasound probe coordinate system, a transformation matrix is ​​determined, specifically including: The least square method is used to perform plane fitting on the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system to obtain the calibration plane equation; The calibration plane equation is converted into a least squares matrix, and the three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are substituted to obtain the normal vector of the calibration plane; Select any two points in the calibration plane to form a plane vector, and construct a calibration plane coordinate system based on the plane vector and the normal vector of the calibration plane; A transformation matrix is ​​determined based on the calibration plane coordinate system and the ultrasound probe coordinate system.

5. The ultrasonic probe calibration method based on data dimension reduction and homography transformation according to claim 4 is characterized in that: The calibration plane equation is shown as follows: Ax+By+Cz+D=0(C≠0); Among them, A, B, C, and D are the coefficients of the calibration plane equation, and (x, y, z) are the three-dimensional coordinates of any calibration point in the ultrasonic probe coordinate system; The calibration plane equation can be transformed into the following formula: z=ax+by+c; in, All are intermediate custom coefficients; The least squares matrix is ​​shown below: b = AX; in, (x i ,y i ,z i ) is the three-dimensional coordinate of the i-th calibration point in the ultrasonic probe coordinate system, and the value range of i is [1,n].

6. The ultrasonic probe calibration method based on data dimension reduction and homography transformation according to claim 5, characterized in that: The transformation matrix is ​​expressed as follows: Among them, M p is the transformation matrix, R is the rotation matrix of the calibration plane coordinate system, t is the translation vector of the origin of the calibration plane coordinate system, t=-Rl1, l1 is the origin of the calibration plane coordinate system.

7. The ultrasonic probe calibration method based on data dimension reduction and homography transformation according to claim 6, characterized in that: The three-dimensional coordinates of each calibration point in the ultrasound probe coordinate system are projected onto the calibration plane according to the following formula: Among them, (x mi ,y mi ,z mi ) is the three-dimensional coordinate of the i-th calibration point in the ultrasound probe coordinate system, (x si ,y si ) is the two-dimensional coordinate of the i-th calibration point on the calibration plane, k i is the depth value of the i-th calibration point on the calibration plane, k i Theoretically it is 0.

8. The ultrasonic probe calibration method based on data dimension reduction and homography transformation according to claim 7, characterized in that: The homography change matrix is ​​calculated according to the following formula: Among them, α is the scale transformation factor, M h is the homography transformation matrix, (x oi ,y oi ) is the two-dimensional coordinate of the i-th calibration point on the ultrasound image.

9. The ultrasonic probe calibration method based on data dimension reduction and homography transformation according to claim 1, characterized in that: According to the transformation matrix and the homography change matrix, converting the two-dimensional coordinates of any target point on the ultrasound image into the three-dimensional coordinates of the target point in the ultrasound probe coordinate system specifically includes: Acquire the two-dimensional coordinates of any target point on the ultrasound image, and calculate the two-dimensional coordinates of the target point on the calibration plane according to the two-dimensional coordinates of the target point on the ultrasound image and the homography change matrix; The three-dimensional coordinates of the target point in the ultrasound probe coordinate system are calculated based on the two-dimensional coordinates of the target point on the calibration plane and the transformation matrix.

10. An ultrasonic probe calibration device based on data dimension reduction and homography transformation, characterized in that: include: Binocular camera, ultrasonic probe, N-line calibration tooling and data processing equipment; visual markers are installed on the ultrasonic probe and the N-line calibration tooling, and the binocular camera can be used to track and obtain the position and posture of the ultrasonic probe and the N-line calibration tooling; the N-line calibration tooling is a three-layer N-line calibration tooling based on binary coding; The data processing device is used to: select a number of calibration points on the ultrasonic image acquired by the ultrasonic probe, and calculate the two-dimensional coordinates of each calibration point on the ultrasonic image based on the N-line calibration tooling; based on the posture relationship between the binocular camera, the ultrasonic probe and the N-line calibration tooling, convert the two-dimensional coordinates of each calibration point to the ultrasonic probe coordinate system to obtain the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system; determine the calibration plane coordinate system based on the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system, and determine the transformation matrix based on the calibration plane coordinate system and the ultrasonic probe coordinate system; project the three-dimensional coordinates of each calibration point in the ultrasonic probe coordinate system into the calibration plane according to the transformation matrix, and calculate the homography change matrix according to the two-dimensional coordinates of each calibration point on the calibration plane and the two-dimensional coordinates of each calibration point on the ultrasonic image; store the transformation matrix and the homography change matrix to complete the calibration of the ultrasonic probe; according to the transformation matrix and the homography change matrix, the two-dimensional coordinates of any target point on the ultrasonic image can be converted into the three-dimensional coordinates of the target point in the ultrasonic probe coordinate system.

Images