A method and device for spatial registration of three-dimensional volume data

Abstract

The present invention relates to the field of computer vision and image processing, and specifically to a method and device for spatial registration of three-dimensional volume data, the method comprising: S1, obtaining the first and second three-dimensional volume data of the same object; S2, roughly registering the first and second three-dimensional volume data by principal component analysis; S3, respectively obtaining the centroid points of each section perpendicular to the first direction of the two three-dimensional volume data after the rough registration, obtaining the centroid point set of each three-dimensional volume data, and obtaining the direction vector after linear fitting of the set; S4, obtaining the registration rotation transformation matrix between the two sets of direction vectors; obtaining the registration translation transformation matrix according to the coordinates of the center centroid point of the two centroid point sets; S5, using the registration rotation transformation matrix and the registration translation transformation matrix to perform coordinate transformation on the second three-dimensional volume data to complete the registration. The present invention makes full use of the geometric space information of the three-dimensional volume data, reduces the size of the data set, shortens the registration time, and improves the registration efficiency and real-time performance.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and image processing, and in particular to a method and device for spatial registration of three-dimensional volume data. Background Art

[0002] With the development of computer vision and image processing, people can quickly and easily obtain volume data containing geometric attribute characteristics and spatial three-dimensional information of objects. Three-dimensional volume data contains rich information about the object itself, and it is increasingly used in many fields such as medical imaging, 3D printing, and gaming and entertainment. Three-dimensional volume data can be collected in a variety of forms, but in the actual collection process, it is limited by the lighting environment, the occlusion of the object itself, the scanning angle of the device, etc., and a single scan often only obtains the three-dimensional volume data of a certain perspective of the object, and there are translation and rotation dislocation phenomena under different perspectives. The spatial matching rule of three-dimensional volume data is aimed at such problems. The three-dimensional volume data under different perspectives are matched to the same coordinate system through a certain geometric transformation rule, so as to facilitate the subsequent analysis and visualization of multiple sets of three-dimensional volume data of the same object under different periods or conditions.

[0003] There are three main types of 3D volume data registration methods in the prior art: the first is a registration method based on geometric feature invariants, which constructs the matching correspondence between points in two sets of 3D volume data through the morphological and structural features of the measured object itself. Its features include contours, shape descriptors, etc., and uses corresponding points to obtain the transformation relationship between the two sets of 3D volume data views, and obtains the registered data through matrix transformation; the second is the traditional iterative closest point method (ICP method) and its improved method, which iteratively searches for the points with the closest Euclidean distance between the target point set and the source point set of the 3D volume data, calculates the transformation matrix between the two, and calculates the matching error until the iterative termination condition is met. The third is a hybrid method, including deep learning, statistical methods, and hybrid methods.

[0004] The defects of the existing technology are mainly as follows: 1. Although the registration method based on geometric feature invariants makes full use of the geometric information of the object, it is more sensitive to the errors of feature extraction and feature matching. Therefore, the final registration result often has large errors, and some registration results are even wrong. 2. The ICP method and its improved method need to provide a better initial registration value, which is highly dependent on the initial registration value, and it is easy to fall into the local optimal solution of the iterative algorithm, and it takes a long time to search for matching point pairs. 3. The hybrid method does not use the original three-dimensional volume data and the three-dimensional volume data to be registered to simplify the point set, and the registration accuracy and speed still need to be further improved. Summary of the invention

[0005] The purpose of the present invention is to overcome the problems in the prior art that the matching process is time-consuming and the matching speed is slow due to the lack of simplified point sets, and to provide a spatial registration method and device for three-dimensional volume data.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0007] A spatial registration method for three-dimensional volume data comprises the following steps:

[0008] S1, acquiring first three-dimensional volume data and second three-dimensional volume data of the same object;

[0009] S2, performing rough registration of the first three-dimensional volume data and the second three-dimensional volume data by using a principal component analysis method;

[0010] S3, respectively obtaining the centroid points of each section perpendicular to the first direction of the two three-dimensional volume data after the rough registration, obtaining a centroid point set of each three-dimensional volume data in the first direction, and obtaining a direction vector after performing a straight line fitting on the centroid point set;

[0011] S4, obtaining a registration rotation transformation matrix between the two sets of direction vectors obtained in step S3; obtaining a registration translation transformation matrix according to the coordinates of the center mass point of the two mass point sets;

[0012] S5, using the registration rotation transformation matrix and the registration translation transformation matrix to perform coordinate transformation on the second three-dimensional volume data to complete the registration.

[0013] Furthermore, the first three-dimensional volume data is original three-dimensional volume data, and the second three-dimensional volume data is target three-dimensional volume data to be registered, and the spatial pixel distance and size of the two are the same.

[0014] Furthermore, step S2 specifically includes: firstly using principal component analysis to process the first three-dimensional volume data and the second three-dimensional volume data to obtain first principal component direction vectors of the two, and then rotating the first three-dimensional volume data and the second three-dimensional volume data to be parallel to the first direction.

[0015] Furthermore, the first direction is along the Z axis, and in step S3, the centroid point (x, y) of each XY slice of the three-dimensional volume data is obtained according to the following formula:

[0016]

[0017] Among them, the coordinate of each pixel point in the X direction on the XY section is x i , the coordinate in the y direction is y i , the corresponding pixel value is p i .

[0018] Furthermore, before performing straight line fitting on the centroid point set in step S3, the coordinates of each centroid point in the set are normalized.

[0019] Furthermore, obtaining the registration rotation transformation matrix between the two sets of direction vectors includes the following steps:

[0020] S401, a direction vector (x) is obtained by using the first set of centroid points of the three-dimensional volume data. o ',y o ',z o ') and the direction vector (x t ',y t ',z t The dot product and cross product between them get the rotation angle θ around the X, Y, and Z axes. x ,θ y ,θ z ;

[0021] S402, by rotating the angle θ around the X, Y, and Z axes x ,θ y ,θ z , get the X, Y, Z direction rotation matrix M x 、M y 、M z ;

[0022] S403, registering the rotation transformation matrix M r =M z *M y *M x .

[0023] Furthermore, the rotation angle θ around the X, Y, and Z axes x ,θ y ,θ z According to the following formulas:

[0024] θ x =atan(((0,y t ',z t ')*(0,y o ',z o ')) / ((0,y t ',z t ')×(0,y o ',z o ')))

[0025] θ y = atan(((x t ',0,z t ')*(x o ',0,z o')) / ((x t ',0,z t ')×(x o ',0,z o ')))

[0026] θ z = atan(((x t ',y t ',0)*(x o ',y o ',0)) / ((x t ',y t ',0)×(x o ',y o ',0))).

[0027] Furthermore, the X, Y, and Z rotation matrices M x 、M y 、M z They are:

[0028]

[0029] Further, obtaining the registration translation transformation matrix according to the coordinates of the center centroid point of the two centroid point sets includes the following steps:

[0030] S411, using the center centroid point C of the first three-dimensional volume data centroid point set o =(x 1 ,y 1 ,z 1 ) and the center centroid point C of the centroid point set of the second three-dimensional volume data t =(x 2 ,y 2 ,z 2 ) to obtain the translation coordinate C:

[0031] C=C 0 -C t =(x 1 -x 2 ,y 1 -y 2 ,z 1 -z 2 );

[0032] S412, obtaining the registration translation transformation matrix M S =[x 1 -x 2 y 1 -y 2 z 1 -z 2 ].

[0033] Based on the same inventive concept, a spatial registration device for three-dimensional volume data is proposed, comprising at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any one of the methods described above.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The method provided by the present invention can simplify the three-dimensional volume data set by using the centroid point of the two-dimensional section according to the three-dimensional volume data of the target to be registered and the original three-dimensional volume data, reduce the dimension of the three-dimensional registration problem to two dimensions, and perform registration transformation on the three-dimensional volume data by calculating the registration rotation transformation matrix and translation transformation matrix of the direction vector and translation coordinates of the three-dimensional volume data of the target to be registered. The present invention makes full use of the geometric space information of the three-dimensional volume data, reduces the size of the data set, shortens the registration time, and improves the registration efficiency and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The present invention is a flow chart of a spatial registration method for three-dimensional volume data.

[0037] Figure 2 Flowchart of the spatial registration method for three-dimensional volume data broken down into some steps.

[0038] Figure 3 The obtained original three-dimensional volume data and the target three-dimensional volume data to be registered.

[0039] Figure 4 It is the direction vector of the original 3D volume data obtained by the PCA method and the target 3D volume data to be registered.

[0040] Figure 5 The three-dimensional volume is displayed after rough alignment.

[0041] Figure 6 This is the display of the normalized centroid point set.

[0042] Figure 7 The direction vector of the centroid is displayed.

[0043] Figure 8 The 3D volume is displayed after registration. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0045] Example 1

[0046] This embodiment takes the registration of carotid artery data as an example to provide a spatial registration method for three-dimensional volume data. Figure 1 As shown, the following steps are included:

[0047] S1, acquiring first three-dimensional volume data and second three-dimensional volume data of the same object;

[0048] S2, performing rough registration of the first three-dimensional volume data and the second three-dimensional volume data by using a principal component analysis method;

[0049] S3, respectively obtaining the centroid points of each section perpendicular to the first direction of the two three-dimensional volume data after the rough registration, obtaining a centroid point set of each three-dimensional volume data in the first direction, and obtaining a direction vector after performing a straight line fitting on the centroid point set;

[0050] S4, obtaining a registration rotation transformation matrix between the two sets of direction vectors obtained in step S3; obtaining a registration translation transformation matrix according to the coordinates of the center mass point of the two mass point sets;

[0051] S5, using the registration rotation transformation matrix and the registration translation transformation matrix to perform coordinate transformation on the second three-dimensional volume data to complete the registration.

[0052] In this embodiment, the Z-axis direction is taken as the first direction, and step S3 and step S4 are further decomposed to provide the following: Figure 2 Detailed steps shown:

[0053] Step 1: Obtain three-dimensional volume data.

[0054] The original three-dimensional volume data Vo and the target three-dimensional volume data Vt to be registered are obtained. The two sets of three-dimensional volume data have the same spatial pixel distance and size and are three-dimensional volume data of the same object obtained at different times or under different conditions. The original three-dimensional volume data Vo is as follows: Figure 3 As shown in the left figure, the target three-dimensional volume data Vt to be registered is as follows Figure 3 As shown in the right figure.

[0055] Step 2: PCA rough registration.

[0056] The original 3D volume data Vo and the target 3D volume data Vt to be registered are processed by the classic PCA method to obtain the first principal component direction vectors of the two, which are (x o ,y o ,z o ) and (x t ,y t ,z t ),as follows Figure 4 As shown, Figure 4The left picture shows the first principal component direction vector of the original 3D volume data Vo. Figure 4 The right figure is the first principal component direction vector of the target three-dimensional volume data Vt to be registered. According to the first principal component direction vector, the original three-dimensional volume data Vo and the target three-dimensional volume data Vt to be registered are rotated to be parallel to the Z axis of the coordinate system, and the three-dimensional volume data Vo' (corresponding to the original three-dimensional volume data Vo) and the three-dimensional volume data Vt' (corresponding to the target three-dimensional volume data Vt to be registered) after rough registration are obtained. The coordinate system is used as the spatial coordinate system of the three-dimensional volume data. The three-dimensional volume data Vo' and Vt' after rough registration are respectively as follows: Figure 5 As shown in the left and right figures.

[0057] Step 3: Obtain a set of scattered points of the geometric space position of the three-dimensional volume data.

[0058] After rough registration of the 3D volume data Vo' and Vt', the geometric centroid points on all XY slices perpendicular to the Z axis are obtained. The specific method is to record the coordinate of each pixel in the X direction on the XY slice as x i , the coordinate in the Y direction is y i , the corresponding pixel value is p i , then the coordinates x of the center of mass in the X direction and the coordinates y of the center of mass in the Y direction are:

[0059]

[0060] The centroid points of the XY section in the Z direction form a set of scattered points in the three-dimensional coordinate system containing the geometric space position information of the three-dimensional volume data, that is, the centroid point set, and the obtained centroid point coordinates are normalized. In this step, the three-dimensional geometric space position centroid point sets of the three-dimensional volume data Vo' and the three-dimensional volume data Vt' are obtained respectively as follows: Figure 6 As shown in the left and right figures.

[0061] Step 4: Determine the direction vector of the three-dimensional volume data.

[0062] According to the three-dimensional centroid point set of the three-dimensional volume data Vo' and the three-dimensional volume data Vt' obtained in step 3, a three-dimensional straight line is fitted to obtain the direction vector (x o ',y o ',z o ') and (x t ',y t ',z t '), respectively represent the spatial direction information of the three-dimensional volume data Vo' and the three-dimensional volume data Vt', respectively. Figure 7 As shown in the left and right figures.

[0063] Step 5: Get the registration rotation transformation matrix.

[0064] According to the two sets of direction vectors (x o ',y o ',z o ') and (x t ',y t ',z t '), calculate the direction vector (x t ',y t ',z t ') and the direction vector (x o ',y o ',z o ') is the angle that needs to be rotated to keep it parallel. The rotation angle θ around the X axis can be obtained by the dot product and cross product of the vectors. x :

[0065] θ x =atan(((0,y t ',z t ')*(0,y o ',z o ')) / ((0,y t ',z t ')×(0,y o ',z o '))),

[0066] Further, obtain the X-direction rotation matrix M x :

[0067]

[0068] Similarly, the rotation angle θ around the Y axis can be obtained y Rotation angle θ around the Z axis z , Y and Z direction rotation matrix M y and M z :

[0069] θ y =atan(((x t ',0,z t ')*(x o ',0,z o ')) / ((x t ',0,z t ')×(x o ',0,z o ')))

[0070] θ z = atan(((x t ',y t ',0)*(x o ',y o ',0)) / ((xt ',y t ',0)×(x o ',y o ',0)))

[0071]

[0072]

[0073] Thus, the registration rotation transformation matrix M is obtained r =M z *M y *M x .

[0074] Step 6: Obtain the registration translation transformation matrix.

[0075] According to the centroid point set of the three-dimensional volume data Vo' obtained in step 3, its center centroid point C is obtained o =(x 1 ,y 1 , z 1 ), and similarly, the center mass point C of the three-dimensional volume data Vt' can be obtained t =(x 2 ,y 2 , z 2 ), find the translation coordinate C = C 0 -C t =(x 1 -x 2 ,y 1 -y 2 , z 1 -z 2 ), obtain the registration translation transformation matrix M s =[x 1 -x 2 y 1 -y 2 z 1 -z 2 ].

[0076] Step 7: 3D volume data registration.

[0077] According to the registration rotation transformation matrix M obtained in step 5 and step 6 r and the registration translation transformation matrix M s , transform the coordinates of the three-dimensional volume data Vt' to the same coordinate system as the three-dimensional volume data Vo', complete the registration of the three-dimensional volume data, and obtain the registered three-dimensional volume data V r The original 3D volume data and the registered 3D volume data are respectively Figure 8 As shown in the left and right figures.

[0078] In addition to the Z-axis direction being the first direction in this embodiment, based on the same principle, in other embodiments, the Y-axis direction can be used as the first direction to obtain the geometric center of mass points of the three-dimensional volume data on all XZ sections perpendicular to the Y-axis direction; or the X-axis direction can be used as the first direction to obtain the geometric center of mass points of the three-dimensional volume data on all YZ sections perpendicular to the X-axis direction.

[0079] The method provided by the present invention uses the centroid point of the two-dimensional section surface to simplify the three-dimensional volume data set according to the three-dimensional volume data of the target to be registered and the original three-dimensional volume data, reduces the dimension of the three-dimensional registration problem to two dimensions, and performs registration transformation on the three-dimensional volume data by calculating the registration rotation transformation matrix and translation transformation matrix of the direction vector and translation coordinates of the three-dimensional volume data of the target to be registered. The present invention makes full use of the geometric space information of the three-dimensional volume data, reduces the size of the data set, shortens the registration time, and improves the registration efficiency and real-time performance.

[0080] Although this embodiment is described by taking the registration of three-dimensional volume data of carotid artery blood vessels as an example, the method provided by the present invention can also be used for the registration of three-dimensional data of the surface geometric shapes of other parts of the human body or animals, medical devices and other general objects in the field of medical imaging; in addition, in addition to the field of medical imaging, the common fields of the method and device of the present invention include reverse engineering, industrial inspection, computer vision, CG production, 3D printing and intelligent manufacturing, and other technical fields involving computer vision and image processing.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A spatial registration method for three-dimensional volume data, characterized in that: The following steps are involved: S1, acquiring first three-dimensional volume data and second three-dimensional volume data of the same object; S2, performing rough registration of the first three-dimensional volume data and the second three-dimensional volume data by using a principal component analysis method; S3, respectively obtaining the centroid points of each section perpendicular to the first direction of the two three-dimensional volume data after the rough registration, obtaining a centroid point set of each three-dimensional volume data in the first direction, and obtaining a direction vector after performing a straight line fitting on the centroid point set; S4, obtaining a registration rotation transformation matrix between the two sets of direction vectors obtained in step S3; obtaining a registration translation transformation matrix according to the coordinates of the center mass point of the two mass point sets; S5, using the registration rotation transformation matrix and the registration translation transformation matrix to perform coordinate transformation on the second three-dimensional volume data to complete the registration.

2. A spatial registration method for three-dimensional volume data according to claim 1, characterized in that: The first three-dimensional volume data is original three-dimensional volume data, and the second three-dimensional volume data is target three-dimensional volume data to be registered, and the spatial pixel distance and size of the two are the same.

3. A spatial registration method for three-dimensional volume data according to claim 1, characterized in that: Step S2 specifically includes: firstly using principal component analysis to process the first 3D volume data and the second 3D volume data respectively to obtain first principal component direction vectors of the two, and then rotating the first 3D volume data and the second 3D volume data to be parallel to the first direction.

4. A spatial registration method for three-dimensional volume data as claimed in claim 3, characterized in that: The first direction is along the Z axis, and in step S3, the centroid point (x, y) of each XY slice of the three-dimensional volume data is obtained according to the following formula: Among them, the coordinate of each pixel point in the X direction on the XY section is x i , the coordinate in the Y direction is y i , the corresponding pixel value is p i .

5. A spatial registration method for three-dimensional volume data according to claim 4, characterized in that: Before performing straight line fitting on the centroid point set in step S3, the coordinates of each centroid point in the set are normalized.

6. A spatial registration method for three-dimensional volume data according to claim 5, characterized in that: Obtaining the registration rotation transformation matrix between two sets of direction vectors includes the following steps: S401, a direction vector (x) is obtained by using the first set of centroid points of the three-dimensional volume data. o ',y o ',z o ') and the direction vector (x t ',y t ',z t The dot product and cross product between them get the rotation angle θ around the X, Y, and Z axes. x ,θ y ,θ z , respectively, according to the following formulas: θ x =where(((0,y t ',z t ’)*(0,y o ',z o ')) / ((0,y t ',z t ' ) × ( 0, y o ',z o '))) θ y =here(((x t ',0,z t ’)*(x o ',0,z o ')) / ((x t ',0,z t ' ) × ( x o ',0,z o '))) θ z =atan(((x t ',and t ',0)*(x o ',and o ',0)) / ((x t ',and t ',0)×(x o ',and o ',0))); S402, by rotating the angle θ around the X, Y, and Z axes x ,θ y ,θ z , get the X, Y, Z direction rotation matrix M x 、M y 、M z ; S403, registering the rotation transformation matrix M r =M z *M y *M x .

7. A spatial registration method for three-dimensional volume data according to claim 6, characterized in that: X, Y, Z direction rotation matrix M x 、M y 、M z They are:

8. A method for spatial registration of three-dimensional volume data according to any one of claims 1 to 7, characterized in that: Obtaining the registration translation transformation matrix according to the coordinates of the center centroid point of the two centroid point sets includes the following steps: S411, using the center centroid point C of the first three-dimensional volume data centroid point set o = (x1, y1, z1) and the center centroid point C of the second three-dimensional volume data centroid point set t =(x2,y2,z2) coordinates, find the translation coordinate C: C=C0-C t =(x1-x2,y1-y2,z1-z2); S412, obtaining the registration translation transformation matrix M S =[x1-x2 y1-y2 z1-z2].

9. A spatial registration device for three-dimensional volume data, characterized in that: The invention comprises at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of claims 1 to 8.

Images