A human body bone deformity measurement method based on bone template deformation matching

Abstract

The application discloses a human body bone deformity measuring method based on bone template deformation matching, and comprises the following steps: constructing a bone template and pre-labeling semantic regions; training a neural network according to the bone template; obtaining a target bone; deforming and fitting the bone template to the target bone by using the trained neural network to obtain a deformed bone template; mapping the pre-labeled semantic regions on the deformed bone template to the target bone to obtain semantic regions of the target bone; and extracting a bone axis according to the semantic regions of the target bone and measuring bone parameters. The application can meet the actual demand of bone parameter measurement and effectively improve the preoperative diagnosis and planning efficiency of bone deformity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bone measurement, in particular to a human body bone deformity measurement method based on bone template deformation matching. BACKGROUND

[0002] In the diagnosis and treatment of bone deformity, doctors need to accurately segment and measure the parameters of the bone before the operation, formulate appropriate surgical plans according to the degree of deformity, reduce the operation time and reduce postoperative complications. The outer surface of the bone constitutes the shape structure of the bone, and the density is high. The internal area of the bone is soft in texture. In the preoperative planning of bone surgery, the doctor needs to consider the position and shape of each semantic region in the bone, and needs to focus on the segmentation and extraction of the outer surface of the bone. To measure the diagnosis and treatment parameters in the bone, such as femoral anteversion angle and neck shaft angle, the semantic region of the bone needs to be extracted first. At present, the extraction of each semantic region of the bone is generally realized by manual segmentation by doctors according to their own clinical experience. This method has low automation level and individual differences, and cannot meet the actual needs of preoperative planning of bone surgery. How to accurately extract the semantic region of the bone has become a difficult problem. SUMMARY

[0003] The purpose of the present application is to provide a human body bone deformity measurement method based on bone template deformation matching, to solve the problem of low accuracy of the method of manually segmenting and extracting each semantic region of the bone by doctors in the prior art.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0005] The present application provides a human body bone deformity measurement method based on bone template deformation matching, which comprises the following steps:

[0006] Construct a bone template and pre-label semantic regions;

[0007] Train a neural network according to the bone template;

[0008] Obtain a target bone;

[0009] Deform and fit the bone template to the target bone using the trained neural network to obtain a deformed bone template;

[0010] Map the pre-labeled semantic regions on the deformed bone template to the target bone to obtain the semantic regions of the target bone;

[0011] Extract the bone axis according to the semantic regions of the target bone and measure the bone parameters.

[0012] Further, the method of constructing the bone template comprises:

[0013] A series of shape vectors V = {X1, X2,..., XN} are obtained from a bone training sample, the bone training sample comprising N human normal bone samples, wherein V is a shape vector of all human normal bone samples in the bone training sample, X1 is a shape vector of a first human normal bone sample, X2 is a shape vector of a second human normal bone sample, and XN is a shape vector of an Nth human normal bone sample. N} are obtained from a bone training sample, the bone training sample comprising N human normal bone samples, wherein V is a shape vector of all human normal bone samples in the bone training sample, X1 is a shape vector of a first human normal bone sample, X2 is a shape vector of a second human normal bone sample, and XN is a shape vector of an Nth human normal bone sample. N} are obtained from a bone training sample, the bone training sample comprising N human normal bone samples, wherein V is a shape vector of all human normal bone samples in the bone training sample, X1 is a shape vector of a first human normal bone sample, X2 is a shape vector of a second human normal bone sample, and XN is a shape vector of an Nth human normal bone sample.

[0014] All shape vectors are normalized wherein Xi is a shape vector of an ith human normal bone sample, u is an average shape vector of all human normal bone samples, and D is a sum of distances between shape vectors of all human normal bone samples and the average shape vector. i wherein Xi is a shape vector of an ith human normal bone sample, u is an average shape vector of all human normal bone samples, and D is a sum of distances between shape vectors of all human normal bone samples and the average shape vector.

[0015] All normalized shape vectors are analyzed and reduced in dimension to obtain a principal component matrix U.

[0016] A bone template s = u + Ua is generated based on the principal component matrix U, wherein s is the bone template, and a is a coefficient vector.

[0017] Further, the method for training the neural network according to the bone template comprises: obtaining different bone templates by modifying the coefficient vector a in the bone template, pre-labeling semantic regions for each bone template, and taking the bone template pre-labeled with the semantic regions as training data of the neural network.

[0018] Further, the method for obtaining the target bone comprises: reconstructing CT data of a patient bone sample into a three-dimensional bone model by using Mimics software, and performing mesh optimization and re-meshing operation on the three-dimensional bone model by using a three-dimensional modeling software 3-matic to obtain the target bone.

[0019] Further, the method for deforming and fitting the bone template to the target bone by using the trained neural network to obtain the deformed bone template comprises:

[0020] All fitting landmark points of the bone template and the target bone are obtained, and a sparse correspondence relationship between all fitting landmark points of the bone template and the target bone is constructed;

[0021] A posterior shape model is established according to the sparse correspondence relationship between all fitting landmark points of the bone template and the target bone, and an ASM algorithm is used to deform and fit the bone template;

[0022] The above operations are iteratively performed for several times to deform and fit the bone template to the target bone to obtain the deformed bone template.

[0023] Further, the use of a trained neural network to deform the bone template to fit the target bone also requires pre-registration of the bone template and the target bone using 3-matic three-dimensional modeling software.

[0024] Further, the method of mapping the pre-labeled semantic regions on the deformed bone template to the target bone to obtain the semantic regions of the target bone comprises:

[0025] The coordinates of each point in the target bone are denoted as Aj=(x j , y j , z j ), where j is 1, 2,..., m, and the first k points closest to each point in the target bone are found from the deformed bone template and denoted as {M1, M2,..., M k}, where M1, M2 and M k are elements of the set of coordinates of a series of feature points in the bone template, and the semantic regions corresponding to the first k points are denoted as {label1, label2,..., label k};

[0026] The semantic region with the highest frequency of occurrence in {label1, label2,..., label k} is denoted as label, and label is taken as the semantic region of A j . The semantic regions of all points in the target bone are calculated according to the above operation, and the semantic regions of the target bone are obtained.

[0027] Further, the method of extracting the bone axis and measuring the bone parameters according to the semantic regions of the target bone comprises:

[0028] Extracting the bone axis according to the semantic regions of the target bone;

[0029] Determining the reference plane based on the bone axis;

[0030] Measuring the angle according to the bone axis and the reference plane.

[0031] Compared with the prior art, the present application has the following advantages:

[0032] The application provides a human body bone deformity measurement method based on bone template deformation matching. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a general schematic diagram of a human body bone deformity measurement method based on bone template deformation matching provided by an embodiment of the application.

[0034] Figure 2 is a comparison schematic diagram of a neural network preliminary prediction bone semantic region and a segmented semantic region of the application.

[0035] Figure 3 is a bone template matching schematic diagram provided by an embodiment of the application.

[0036] Figure 4 is a bone axis extraction schematic diagram provided by an embodiment of the application.

[0037] Figure 5 is a semantic segmentation specific result schematic diagram provided by an embodiment of the application.

[0038] Figure 6 is a head and neck region amplification result schematic diagram provided by an embodiment of the application.

[0039] Figure 7 is an inner and outer condyle region amplification result schematic diagram provided by an embodiment of the application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the application.

[0041] As shown in Figure 1 is a general schematic diagram of a human body bone deformity measurement method based on bone template deformation matching provided by an embodiment of the application.

[0042] The embodiment of the present application provides a human body bone deformity measurement method based on bone template deformation matching, which comprises the following steps:

[0043] S1: constructing a bone template and pre-labeling semantic regions.

[0044] The specific method for constructing the bone template comprises the following steps.

[0045] S11: obtaining a series of shape vectors from bone training samples.

[0046] The bone training samples comprise N human body normal bone samples, n feature points are manually marked for each human body normal bone sample, and the coordinates of each feature point are recorded to obtain a series of feature point coordinate sets:

[0047] M={(x1,y1,z1),(x2,y2,z2),...,(x n ,y n ,z n )}

[0048] wherein (x w , y w , z w ) is the coordinate of each feature point (w=1, 2...n),

[0049] Each human body normal bone sample can be expressed as a shape vector:

[0050] X=(x1,y1,z1,x2,y2,z2,...,x n ,y n ,z n ) T

[0051] wherein T is the transpose of the vector,

[0052] A series of shape vectors are obtained from the bone training samples:

[0053] V={X1,X2,...,X N}

[0054] wherein V is the shape vector of all human body normal bone samples in the bone training samples, X1 is the shape vector of the first human body normal bone sample, X2 is the shape vector of the second human body normal bone sample, and X N is the shape vector of the Nth human body normal bone sample.

[0055] S12: Normalize all shape vectors, the purpose is to eliminate non-shape interference, eliminate the influence of different feature point positions, eliminate the influence of direction changes and other differences. Shape vector normalization is achieved by rotating, scaling and translating the shape vector. Procrustes algorithm is used to normalize all shape vectors:

[0056]

[0057] Where X i is the shape vector of the i-th human body normal skeleton sample, u is the average shape vector of all human body normal skeleton samples, and D is the sum of distances between all human body normal skeleton samples and the average shape vector.

[0058] S13: Analyze and reduce the dimension of all normalized shape vectors using PCA algorithm. Calculate the average shape vector of all human body normal skeleton samples:

[0059]

[0060] Then further calculate the covariance matrix R:

[0061]

[0062] Where T is the transpose of the vector,

[0063] Calculate the eigenvalue λi of the covariance matrix R and the corresponding eigenvector Ci, arrange them in descending order of eigenvalue, i.e. λ i ≥λ i+1 . Each eigenvalue λ i is the variance of the i-th human body normal skeleton sample in the corresponding eigenvector direction, is the corresponding standard deviation, and the standard deviation is denoted as σi, i.e. In descending order of eigenvalue, take the specified number of eigenvalues and the corresponding eigenvectors to form the principal component matrix U.

[0064] S14: Generate a skeleton template based on the principal component matrix U:

[0065] s = s (α) = u + Uα

[0066] Where s is the skeleton template, α is the coefficient vector, different α corresponds to different skeleton templates, and the coefficient vector α = (b1, b2,..., b i ),

[0067] Where b j is within 3 standard deviations, i.e. satisfies:

[0068]

[0069] Modification b i For different parameter values, i.e. new bone templates can be generated.

[0070] S2: training the neural network according to the bone template, the specific method comprising: obtaining different bone templates by modifying the coefficient vector a in the bone template, pre-labeling each semantic region for each bone template, obtaining a bone template with semantic regions, and taking the bone template with semantic regions as training data for the neural network.

[0071] S3: obtaining the target bone, the specific method comprising: reconstructing the CT data of the patient's bone sample into a three-dimensional bone model using Mimics software, and performing grid optimization and re-meshing operation on the three-dimensional bone model using three-dimensional modeling software 3-matic to obtain the target bone.

[0072] S4: first pre-register the bone template and the target bone using three-dimensional modeling software 3-matic. The purpose of pre-registration is to make the spatial position of the bone template and the target bone consistent, so as to facilitate the deformation fitting operation in the next step. The essence of pre-registration is to perform rigid transformation on the target bone, i.e. translation transformation, rotation transformation and reverse transformation, to change its position in space, and the distance between the two points before and after transformation remains unchanged.

[0073] Then use the trained neural network to deform and fit the bone template to the target bone to obtain the deformed bone template, the specific method comprising:

[0074] S41: obtaining all fitting landmarks of the bone template and the target bone, and constructing the sparse correspondence relationship between all fitting landmarks of the bone template and the target bone, the specific method comprising:

[0075] S411: predicting the corresponding semantic regions of the bone template and the target bone respectively by the trained neural network, for example, taking the femoral head concave, the lesser trochanter, the medial condyle and the lateral condyle regions;

[0076] S412: calculating the center points of the semantic regions corresponding to the bone template and the target bone, for example, taking the center point of the lesser trochanter as P;

[0077] S413: traversing the sampling points on the surface of the lesser trochanter region, calculating the nearest point to the center point of the lesser trochanter in the bone template and the target bone as Q, and taking Q as the fitting landmark of the bone template and the target bone;

[0078] S414: repeating S412 and S413 to obtain all fitting landmarks of the bone template and the target bone, and constructing the sparse correspondence relationship between all fitting landmarks of the bone template and the target bone.

[0079] S42: Based on the sparse correspondence between all fitted marker points of the bone template and the target bone, a posterior shape model is established, and the ASM algorithm (Active Shape Model algorithm) is used to deform and fit the bone template.

[0080] S43: Repeat S41 and S42, iterating through the above operations several times to more accurately deform and fit the bone template to the target bone, obtaining the deformed bone template, such as... Figure 3 As shown.

[0081] S5: Map the pre-annotated semantic regions on the deformed bone template to the target bone to obtain the semantic region of the target bone. This involves mapping the pre-annotated semantic regions on the deformed bone template to the corresponding positions on the target bone, assigning the same semantic regions to the target bone, and thus obtaining the semantic region of the target bone. The semantic region corresponding to each point in the target bone is calculated using the KNN algorithm.

[0082] The semantic regions of the target bone are obtained by mapping the pre-annotated semantic regions on the deformed bone template to the target bone. Specific methods include:

[0083] S51: Mark the coordinates of each point in the target bone as A j =(x j y j , z j ), where j is 1, 2, ..., m, and the k points closest to each point in the target bone are found from the deformed bone template and denoted as {M1, M2, ..., M}. k}, where M1, M2 and M k All are elements in a set M of coordinates of a series of feature points in the skeletal template, and the semantic regions corresponding to the first k points are denoted as {label1, label2, ..., label...}. k};

[0084] S52: Statistical calculation {label1, label2, ..., label...} k The semantic region that appears most frequently in the array is denoted as label, and label is used as A. j semantic regions;

[0085] S53: Repeat S51 and S52 to calculate the semantic regions of all points in the target skeleton, thus obtaining the semantic regions of the target skeleton.

[0086] S6: Extract the skeletal axis based on the semantic region of the target bone, measure skeletal parameters, and determine whether the target bone is deformed. Taking femoral semantic segmentation as an example, measure its neck-shaft angle and anteversion angle. The specific methods include:

[0087] S61: Extract the bone axis according to the semantic region of the target bone, and the bone axis extraction is as shown in Figure 4

[0088] The head-neck axis, femoral shaft axis and medial-lateral condyle connecting line are extracted.

[0089] Head-neck axis extraction: the femoral head region and femoral neck region are obtained by semantic mapping, the center points of the sampling points of the femoral head region and the sampling points of the femoral neck region are respectively calculated, and the connecting line of the two center points is the head-neck axis.

[0090] Femoral shaft axis extraction: the femoral shaft region is obtained by semantic mapping, the sampling points of the femoral shaft region are divided along the horizontal direction and equally divided in the vertical direction, the center points of each sampling point set are calculated, and finally the center points are fitted into a straight line by the least square method, and the femoral shaft axis is obtained.

[0091] Medial-lateral condyle connecting line extraction: the medial-lateral condyle region is obtained by semantic mapping, the outermost points of the medial-lateral condyle region are calculated, and the connecting line of the two outermost points is the medial-lateral condyle connecting line.

[0092] S62: Determine the reference plane based on the bone axis.

[0093] The reference plane is composed of a coronal plane, an axial plane and a sagittal plane, and the three planes are perpendicular to each other; the coronal plane passes through the femoral shaft axis and is parallel to the medial-lateral condyle connecting line; the axial plane is perpendicular to the femoral shaft axis, and the sagittal plane is perpendicular to the coronal plane and the axial plane.

[0094] S63: Measure the angle according to the bone axis and the reference plane.

[0095] After the bone axis extraction and the reference plane determination, the neck-shaft angle and the anteversion angle are measured. Clinically, the normal range of the neck-shaft angle is [125°, 145°], and the normal range of the anteversion angle is [0°, 20°].

[0096] The measurement method of the neck-shaft angle and the anteversion angle is as follows:

[0097] Neck-shaft angle measurement: the neck-shaft angle is the included angle formed by the projection of the head-neck axis and the femoral shaft axis on the coronal plane.

[0098] Anteversion angle measurement: the anteversion angle is the included angle formed by the projection of the head-neck axis and the medial-lateral condyle connecting line on the axial plane.

[0099] The semantic segmentation region predicted by the neural network is the preliminary semantic region, and compared with the real semantic region, the error is large and the accuracy is not high, so further semantic segmentation of the target bone is needed to realize accurate semantic region transmission. The comparison of the preliminary semantic region of the bone predicted by the neural network and the further segmented semantic region of the application is as shown in Figure 2

[0100] Experimental example ​​

[0101] Semantic segmentation and angle measurement were performed on 4 real human femur deformities and 10 real human healthy femurs. The semantic regions of femoral head, femoral neck, femoral shaft, medial condyle, lateral condyle were obtained by semantic segmentation, and the femoral neck-shaft angle and anteversion angle were measured. The real semantic regions of femur were obtained by manual segmentation through 3-matic software, and the real values of neck-shaft angle and anteversion angle were obtained by manual measurement. The semantic regions obtained by the three methods (neural network prediction semantic region, template fitting mapping semantic region, and the method of the present application) were compared with the manually segmented semantic regions, and the angle data was also compared.

[0102] Experiments 1 to 4 were femoral deformities, and experiments 5 to 14 were healthy femurs. The Dice coefficient comparison results obtained by the present application are shown in Table 1, which records the Dice coefficients of the semantic segmentation of femur obtained by the neural network prediction semantic region, the template fitting mapping semantic region, and the semantic region segmentation method of the present application, respectively. The average Dice coefficient is the average value of the Dice coefficients of each semantic region.

[0103] Table 1 Detailed results analysis and comparison of semantic segmentation Dice coefficients

[0104]

[0105]

[0106] The angle data comparison results are shown in Table 2. The table records the angle data measured by the above three methods and the angle data measured by manual measurement.

[0107] Table 2 Detailed results analysis and comparison of angle measurement data

[0108]

[0109]

[0110]

[0111] The semantic segmentation results are shown in Figure 5 The figure records the semantic regions obtained by manual annotation, neural network prediction, template fitting mapping, and the method of the present application. The enlarged results of the head-neck region are shown in Figure 6 The enlarged results of the medial and lateral condyle regions are shown in Figure 7

[0112] ​Dice coefficient result analysis: the average Dice coefficient obtained by predicting the semantic region by the neural network is 0.85836, the average Dice coefficient obtained by fitting and mapping the semantic region by the template is 0.84649, and the average Dice coefficient of the semantic region obtained by the method is 0.94015, and the Dice coefficient of the semantic region segmented by the method is higher than that of the other two methods.

[0113] Angle measurement data result analysis: the error of the angles measured by the above three methods and the angles measured by the manual method is compared by the standard deviation δ, and the calculation method is:

[0114]

[0115] Wherein, H is the number of experimental bones, and θ d is the angle value measured by the above three methods for the dth experimental bone, is the angle value measured by the manual method for the dth experimental bone.

[0116] The standard deviation result is analyzed, and the accuracy of the angles measured by the three methods is compared. The angles of the semantic region predicted by the neural network are measured, and the angles measured by the manual method are calculated, the standard deviation of the cervico-dural angle is 7.9496°, and the standard deviation of the forward inclination angle is 5.5268°; the angles of the semantic region obtained by template fitting and mapping are measured, and the angles measured by the manual method are calculated, the standard deviation of the cervico-dural angle is 8.6083°, and the standard deviation of the forward inclination angle is 6.7785°; the angles of the semantic region segmented by the method are measured, and the angles measured by the manual method are calculated, the standard deviation of the cervico-dural angle is 3.1725°, and the standard deviation of the forward inclination angle is 2.6190°.

[0117] From the experimental results, it can be seen that the human body bone deformity measurement method based on bone template deformation matching provided by the application is better than the other two methods, and compared with the parameter data results obtained by manual measurement, the error is within an acceptable range.

[0118] Compared with the prior art, the method has the following advantages: first, the fitting mark points are no longer manually selected, but are predicted by a neural network, the correspondence of the selected fitting mark points is more accurate, and the degree of automation is improved; second, the bone template is iteratively fitted to the target bone, the bone template deformation is more accurate, and accurate semantic mapping is achieved.

[0119] It will be apparent to those skilled in the art that the application is not limited to the details of the above-exemplified embodiments and that the present application can be implemented in other particular forms without departing from the spirit or essential characteristics of the present application. The embodiments should therefore be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the above description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. No reference signs in the claims should be considered as limiting the scope of the claims with respect to the figures of the patent document.

Claims

1. A method of measuring human skeletal deformities based on skeletal template morphing matching, characterized by, The method comprises the following steps: constructing a bone template and pre-labeling semantic regions; training a neural network according to the bone template; obtaining a target bone; deforming and fitting the bone template to the target bone by using the trained neural network to obtain a deformed bone template, comprising: obtaining all fitting landmark points of the bone template and the target bone, and constructing a sparse correspondence relationship between all fitting landmark points of the bone template and the target bone; establishing a posterior shape model according to the sparse correspondence relationship between all fitting landmark points of the bone template and the target bone, and deforming and fitting the bone template by using an ASM algorithm; iteratively performing the above operations for several times to deform and fit the bone template to the target bone to obtain a deformed bone template; mapping the pre-labeled semantic regions on the deformed bone template to the target bone to obtain semantic regions of the target bone, comprising: Let the coordinates of each point in the target skeleton be denoted as where j is 1, 2, …, m, and the first k points closest to each point in the target skeleton are found from the deformed skeleton template and denoted as where , and are elements in the set M of coordinates of a series of feature points in the skeleton template, and the semantic region to which the first k points correspond is denoted as ; Statistical calculation The semantic region with the highest occurrence frequency in the middle is recorded as , the semantic region of the target bone is calculated according to the above operation, and the semantic region of the target bone is obtained. As the semantic region, the semantic region of all points in the target bone is calculated according to the above operation, and the semantic region of the target bone is obtained. extracting a bone axis according to the semantic regions of the target bone, and measuring bone parameters.

2. The human body bone deformity measuring method based on the skeleton template morphing matching according to claim 1, characterized in that, The method for constructing a bone template comprises: obtaining a series of shape vectors from a skeletal training sample , the skeletal training sample comprising N human normal skeletal samples, wherein, is a shape vector of all human normal skeletal samples in the skeletal training sample, is a shape vector of the first human normal skeletal sample, is a shape vector of the second human normal skeletal sample, is a shape vector of the Nth human normal skeletal sample; normalizing all shape vectors wherein, is the shape vector of the first personal body normal bone sample, is the average shape vector of all personal body normal bone samples, is the sum of distances between the shape vector of the first personal body normal bone sample and the average shape vector of all personal body normal bone samples. Analyzing and reducing dimensionality on all normalized shape vectors, get principal component matrix ; Based on a principal component matrix Generating a bone template wherein, is a bone template, is a coefficient vector.

3. The human body bone deformity measuring method based on the skeleton template morphing matching according to claim 2, characterized in that, The method for training the neural network according to the bone template comprises: obtaining different bone templates by modifying the coefficient vector in the bone template, pre-labeling semantic regions for each bone template, and taking the bone template pre-labeled with the semantic regions as the training data of the neural network. , obtaining different bone templates by modifying the coefficient vector in the bone template, pre-labeling semantic regions for each bone template, and taking the bone template pre-labeled with the semantic regions as the training data of the neural network.

4. The method of claim 1, wherein the method further comprises: The method for obtaining a target bone comprises: reconstructing CT data of a patient's bone sample into a three-dimensional bone model by using Mimics software, and then performing mesh optimization and re-meshing operations on the three-dimensional bone model by using a three-dimensional modeling software 3-matic to obtain a target bone.

5. The method of claim 1, wherein the method further comprises: Before deforming and fitting the bone template to the target bone by using the trained neural network, the bone template and the target bone need to be pre-registered by using a three-dimensional modeling software 3-matic.

6. The method of claim 1, wherein the method further comprises: The method for extracting a bone axis according to the semantic regions of the target bone and measuring bone parameters comprises: extracting a bone axis according to the semantic regions of the target bone; determining a reference surface based on the bone axis; measuring an angle according to the bone axis and the reference surface.

Images