A fully automatic measurement method, system, terminal and medium for knee varus and valgus

Abstract

The present invention discloses a fully automatic measurement method, system, terminal and medium for knee varus and valgus, relating to the technical field of image processing. The key points of its technical solution are as follows: obtaining three-dimensional lower limb bone data; after segmenting and reconstructing the lower limb bones based on the three-dimensional lower limb bone data, obtaining a three-dimensional lower limb bone network model, and extracting the surface data points of the lower limb bones; extracting key points including the center point of the femoral head, the center point of the knee joint, and the center point of the ankle joint surface from the surface data points of the lower limb bones, and determining the femoral mechanical axis and the tibial mechanical axis based on the key points; performing an inverse cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to calculate the HKA angle. Based on prior knowledge, the present invention directly extracts key points from the segmentation results of the lower limb bones. Its processing speed will not increase significantly with the increase in data volume, requires less computing power, takes less time, and has higher efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, it relates to a fully automatic measurement method, system, terminal and medium for knee varus and valgus. Background Art

[0002] The hip-knee-ankle (HKA) angle refers to the angle between the mechanical axis of the femur and the mechanical axis of the tibia, and is an important indicator for evaluating knee varus and valgus deformities. Among them, the mechanical axis of the femur refers to the line connecting the center of the femoral head to the center of the knee joint, and the mechanical axis of the tibia refers to the line connecting the center of the knee joint to the center of the ankle joint.

[0003] The existing fully automatic measurement methods for knee varus and valgus mainly include X-ray film detection methods and three-dimensional image detection methods based on deep learning. In the X-ray film detection method, a deep learning network or a traditional method is generally used to segment or detect key points of the lower limb in the X-ray film; since the X-ray film is a two-dimensional projection imaging, and its important quality standard is that the patella is centered between the femoral condyles, the patient's standing position and leg position have a great influence on the measurement result of knee varus and valgus, and it is easy to cause measurement errors due to factors such as the patient himself or the technician's operation. In the three-dimensional image detection method based on deep learning, a deep learning network is generally used to predict key points of the lower limb in the three-dimensional image; since the data volume of the three-dimensional image data is much larger than that of the two-dimensional image data, in order to detect multiple key points globally, the deep learning network needs to perform a large amount of operations and takes a long time. Summary of the Invention

[0004] To solve the deficiencies in the prior art, the purpose of the present invention is to provide a fully automatic measurement method, system, terminal and medium for knee varus and valgus, which directly extracts key points from the segmentation result of the lower limb bones based on prior knowledge, and determines the mechanical axis of the femur and the mechanical axis of the tibia according to the key points, and then performs an inverse cosine solution on the dot product between the unit vector of the mechanical axis of the femur and the unit vector of the mechanical axis of the tibia, and the HKA angle can be obtained. Its processing speed will not increase significantly with the increase of the data volume, the requirement for computing power is smaller, the time consumption is less, and the efficiency is higher.

[0005] The above technical purpose of the present invention is achieved through the following technical solutions:

[0006] In the first aspect of the present invention, a fully automatic measurement method for knee varus and valgus is provided, including the following steps:

[0007] Obtain three-dimensional lower limb bone data;

[0008] After segmenting and reconstructing the lower limb bones according to the three-dimensional lower limb bone data, a three-dimensional lower limb bone network model is obtained, and the surface data points of the lower limb bones are extracted;

[0009] Extract key points including the center point of the femoral head, the center point of the knee joint, and the center point of the ankle joint surface from the surface data points of the lower limb bones, and determine the femoral mechanical axis and the tibial mechanical axis based on the key points;

[0010] Perform inverse cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to calculate the HKA angle.

[0011] In one implementation, the segmentation of the lower limb bones is achieved by threshold segmentation or deep learning segmentation methods.

[0012] In one implementation, the segmentation of the lower limb bones is achieved by deep learning segmentation methods, including:

[0013] Train a deep learning segmentation network for three-dimensional lower limb bones;

[0014] Perform binary classification on the pixels in the three-dimensional space by the deep learning segmentation network according to whether they are inside the lower limb bone region; if the pixel is in the non-lower limb bone region, it is assigned a value of 0; if the pixel is in the lower limb bone region, it is assigned a value of 1;

[0015] Take the classification result of the binary classification as the three-dimensional lower limb bone voxel model.

[0016] In one implementation, the training of the deep learning segmentation network for three-dimensional lower limb bones includes:

[0017] Perform a coronal forward projection on the full lower limb sequence composed of multiple tomographic images to form a two-dimensional forward projection image;

[0018] Use a two-dimensional U-Net to perform rough segmentation on the two-dimensional forward projection image to roughly locate each bone;

[0019] Put the tomographic images into a three-dimensional U-Net in groups of multiple layers of images for fine bone segmentation, and refer to the results of roughly locating each bone during segmentation.

[0020] In one implementation, the method further includes:

[0021] There are overlapping tomographic images between adjacent groups of images;

[0022] After the segmentation of adjacent groups of images is completed, perform intersection fusion on the segmentation results of the overlapping tomographic images; if the bone voxel appears in the overlapping area of both groups, the bone voxel is retained, otherwise it is deleted.

[0023] In one implementation, the reconstruction of the lower limb bones is achieved by extracting the isosurface or registering the deficit model.

[0024] In one implementation, the reconstruction of the lower limb bones is achieved by extracting the isosurface, including:

[0025] Set the threshold of the desired isosurface as M;

[0026] Construct a plurality of cube voxels of size N×N×N centered on each pixel in the three-dimensional lower limb bone voxel model obtained after segmentation;

[0027] Compare the pixel values at the vertices of the cube voxel with the threshold M, and screen out the first voxels that intersect with the isosurface;

[0028] Find the intersection points of the isosurface and the edges of the first voxels by interpolation, and connect the intersection points into triangles to form isosurface patches. The three-dimensional lower limb bone network model is composed of the set of triangles in all the first voxels.

[0029] In one implementation, the extraction of the surface data points of the lower limb bones includes:

[0030] Perform connectivity detection on the three-dimensional lower limb bone network model to obtain a plurality of non-connected connected domains;

[0031] Calculate the central coordinates of each of the connected domains;

[0032] Classify the lower limb bones into the left femur, right femur, left tibia, right tibia, left talus, and right talus according to the central coordinates of the connected domains, and retain the surface data points of each of the lower limb bones.

[0033] In one implementation, the calculation of the central coordinates of each of the connected domains includes:

[0034] Perform binary classification on each pixel in the three-dimensional space according to whether it is inside the connected domain, and statistically calculate the average coordinate of all the pixels inside the connected domain to obtain the central coordinate of the connected domain.

[0035] In one implementation, the method further includes:

[0036] Set a volume threshold;

[0037] Calculate the volume of each connected domain, where the volume of the connected domain is the sum of the number of pixels inside the connected domain;

[0038] If the volume of the connected domain is greater than the volume threshold, the corresponding connected domain is identified as a valid connected domain; and if the volume of the connected domain is less than the volume threshold, the corresponding connected domain is identified as noise.

[0039] In one implementation, the extraction process of the center point of the femoral head is specifically as follows:

[0040] Select the data points of the proximal femur and the medial femur as the femoral head data points;

[0041] Select the femoral head data point with the highest height as the femoral head vertex and the femoral head data point closest to the medial side of the human body as the medial femoral head point, and determine the initial femoral head center by combining the Z-axis coordinate of the medial femoral head point and the X-axis and Y-axis coordinates of the femoral head vertex;

[0042] Calculate the first distances from the initial femoral head center to the femoral head vertex and the medial femoral head point respectively, and take the average of the two calculated first distances as the rough estimated radius of the femoral head;

[0043] Calculate the second distance from the femoral head data point to the initial femoral head center, and perform spherical fitting on the femoral head data points with the second distance less than 1.2 times the rough estimated radius to obtain the accurately determined femoral head center point.

[0044] In one implementation, the extraction process of the femoral head center point is specifically as follows:

[0045] Select cross-sections at multiple different positions or directions of the proximal femur, and respectively detect the center of the two-dimensional circular femoral head structure within the cross-sections;

[0046] And calculate the center of the sphere in three-dimensional space using multiple centers to obtain the femoral head center.

[0047] In one implementation, the extraction process of the knee joint center point is specifically as follows:

[0048] Select the data points of the proximal tibia as the knee joint data points;

[0049] Select the knee joint data point closest to the medial side of the human body as the medial knee joint point and the knee joint data point closest to the lateral side of the human body as the lateral knee joint point;

[0050] Take the average of the coordinates of the medial knee joint point and the lateral knee joint point to obtain the knee joint center point.

[0051] In one implementation, the extraction process of the knee joint center point is specifically as follows:

[0052] Select the data points of the proximal tibia as the knee joint data points;

[0053] Take the average of the coordinates of all the knee joint data points to obtain the knee joint center point.

[0054] In one implementation, the extraction process of the ankle joint surface center point is specifically as follows:

[0055] Select the data points at the proximal end of the talus as the ankle joint data points;

[0056] Select the ankle joint data point closest to the medial side of the human body as the medial ankle joint point and the ankle joint data point closest to the lateral side of the human body as the lateral ankle joint point;

[0057] Average the coordinates of the medial ankle joint point and the lateral ankle joint point to obtain the center point of the ankle joint.

[0058] In one implementation, the center point of the ankle joint surface uses the center points of the medial and lateral malleoli.

[0059] In one implementation, if the HKA angle is an angle in two dimensions, the unit vectors of the femoral mechanical axis and the tibial mechanical axis are both composed of the coordinates of the X-axis and the Z-axis;

[0060] And / or, if the HKA angle is an angle in three dimensions, the unit vectors of the femoral mechanical axis and the tibial mechanical axis are both composed of the coordinates of the X-axis, the Y-axis, and the Z-axis.

[0061] In a second aspect of the present invention, there is provided a fully automatic measurement system for knee joint varus and valgus, which is used to implement a fully automatic measurement method for knee joint varus and valgus provided in the first aspect of the present invention, including:

[0062] A data acquisition module for acquiring three-dimensional lower limb bone data;

[0063] A surface extraction module for segmenting and reconstructing the lower limb bone based on the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model and extracting the surface data points of the lower limb bone;

[0064] A center detection module for extracting key points including the center point of the femoral head, the center point of the knee joint, and the center point of the ankle joint surface from the surface data points of the lower limb bone, and determining the femoral mechanical axis and the tibial mechanical axis based on the key points;

[0065] An angle calculation module for performing an inverse cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to calculate the HKA angle.

[0066] In a third aspect of the present invention, there is provided a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a fully automatic measurement method for knee joint varus and valgus provided in the first aspect of the present invention.

[0067] In a fourth aspect of the present invention, there is provided a computer-readable medium having stored thereon a computer program, which when executed by a processor can implement a fully automatic measurement method for knee varus and valgus as provided in the first aspect of the present invention.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] 1. The fully automatic measurement method for knee varus and valgus provided by the present invention directly extracts key points from the segmentation result of the lower limb bones based on prior knowledge, determines the femoral mechanical axis and the tibial mechanical axis according to the key points, and then performs an inverse cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to obtain the HKA angle. Its processing speed will not increase significantly with the increase of data volume, requires less computing power, takes less time, and has higher efficiency;

[0070] 2. The deep learning model described in the present invention can not only be used for traditional spiral CT data, but also be applied to low-dose cone beam CT data; the dose of low-dose cone beam CT used in the deep learning model of the present invention is equivalent to 10% - 20% of the dose of traditional spiral CT, and the present invention can have a good segmentation effect on bone tissue, and different bones can be automatically classified and recognized;

[0071] 3. The present invention can not only calculate the HKA angle in three dimensions, but also be applicable to the calculation of the HKA angle in two dimensions, and its application flexibility is relatively strong;

[0072] 4. The present invention realizes the full-automatic measurement of knee varus and valgus, solves the problem of high positioning requirements caused by two-dimensional projection in the previous X-ray film detection method, solves the problems of manpower consumption and measurement error caused by three-dimensional complex structures in the three-dimensional structure manual annotation method, and solves the problem of time-consuming caused by large data volume in the three-dimensional image data automatic detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, form a part of this application, and do not limit the embodiments of the present invention. In the drawings:

[0074] Figure 1 is the flowchart in Embodiment 1 of the present invention;

[0075] Figure 2 is the flowchart of surface extraction in Embodiment 1 of the present invention;

[0076] Figure 3 is the coronal plane of the fusion diagram of the image and bone segmentation result in the prior art;

[0077] Figure 4is the cross-section of the fused image and bone segmentation results in the prior art;

[0078] Figure 5 is the sagittal plane of the fused image and bone segmentation results in the prior art;

[0079] Figure 6 is the 3D rendering of the bone segmentation results in the prior art;

[0080] Figure 7 is the coronal plane of the fused image and bone segmentation results in Example 1 of the present invention;

[0081] Figure 8 is the cross-section of the fused image and bone segmentation results in Example 1 of the present invention;

[0082] Figure 9 is the sagittal plane of the fused image and bone segmentation results in Example 1 of the present invention;

[0083] Figure 10 is the 3D rendering of the bone segmentation results in Example 1 of the present invention;

[0084] Figure 11 is the visualization schematic diagram of the lower limb bones and the HKA angle in Example 1 of the present invention;

[0085] Figure 12 is the system block diagram in Example 2 of the present invention. Detailed implementation manners

[0086] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in combination with embodiments and drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0087] Example 1: A fully automatic measurement method for knee varus and valgus, as Figure 1 shown, includes the following steps:

[0088] S1: Obtain three-dimensional lower limb bone data;

[0089] S2: After segmenting and reconstructing the lower limb bones based on the three-dimensional lower limb bone data, obtain a three-dimensional lower limb bone network model, and extract the surface data points of the lower limb bones;

[0090] S3: Extract key points including the center point of the femoral head, the center point of the knee joint, and the center point of the ankle joint surface from the surface data points of the lower limb bones, and determine the mechanical axis of the femur and the mechanical axis of the tibia based on the key points;

[0091] S4: Perform an inverse cosine solution on the dot product between the unit vector of the mechanical axis of the femur and the unit vector of the mechanical axis of the tibia to calculate the HKA angle.

[0092] In step S1, the three-dimensional lower limb bone data can be separated from the three-dimensional data, and the three-dimensional data can be composed of voxels, point clouds or meshes, such as CT (Computed Tomography) scan data and MRI (Magnetic Resonance Imaging) data for voxels.

[0093] In step S2, the segmentation of the lower limb bone can be achieved by the method of threshold segmentation or by the method of deep learning segmentation.

[0094] In addition, the reconstruction of the lower limb bone can be achieved by the method of extracting the isosurface or by the method of sparse model registration.

[0095] Taking the three-dimensional CT scan data of the human body as an example, as Figure 2 shown, it is specifically implemented by the following steps.

[0096] (1) Input the three-dimensional CT scan data of the human body.

[0097] (2) Deep learning segmentation: Train the deep learning segmentation network for the three-dimensional lower limb bone, perform binary classification on each pixel in the three-dimensional space according to whether it is inside the lower limb bone area, and assign values of 0 or 1 respectively, where 0 represents the non-lower limb bone area and 1 represents the lower limb bone area. The classification result is the three-dimensional lower limb bone voxel model.

[0098] For the CT sequence of the three-dimensional lower limb bone, a layer interval of 1.0 mm and a layer thickness of 2.0 mm are generally used. The tomogram is a 512×512 matrix. For patients of general height, there are about 1000 lower limb images, and the storage capacity is about 1 GB. Directly feeding them into the AI model for training or inference at one time will consume a large amount of computer (memory and GPU video memory) resources. Directly feeding the entire case into the AI model will increase the development and usage costs. While feeding the tomographic images into the AI model in batches will cause the AI model to be unable to obtain global information. Therefore, the present invention adopts a two-stage AI model architecture and training method.

[0099] First, perform a forward coronal projection using the full lower limb sequence of the entire patient (about 1000 tomographic images, equivalent to a spatial length of 1000 mm along the Z-axis) to form a two-dimensional forward projection image. First, use a two-dimensional U-Net to roughly segment the two-dimensional forward projection image and roughly locate each bone. Then, group the tomographic images in sets of 32 and put them into a three-dimensional U-Net for fine bone segmentation. During segmentation, the bone location information from the two-dimensional U-Net will be referred to. This global plus local segmentation method can not only avoid excessive consumption of computer resources but also enable the model to fully consider global information and prevent problems of mis-segmentation and misclassification. The model obtained by initially segmenting the three-dimensional lower limb bone data, namely the three-dimensional lower limb bone voxel model, can then be reconstructed on the basis of the three-dimensional lower limb bone voxel model to obtain the three-dimensional lower limb bone network model.

[0100] In addition, to avoid misalignment when the segmentation results of adjacent groups are stitched together, an overlap is set between groups, generally 1 / 4 of the number of images in a group. For example, if there are 32 images in a group, then 8 images of the current group overlap with the previous group, and 8 images of the current group overlap with the next group. After the adjacent groups are segmented, the segmentation results of the overlapping part are fused by taking the intersection, that is, the identified bone voxels can only be retained if they appear in the overlapping areas of both groups, otherwise they are deleted.

[0101] The present invention can be applied not only to traditional spiral CT data but also to low-dose cone beam CT data. The dose of the low-dose cone beam CT used in the deep learning model of the present invention is equivalent to 10% - 20% of the dose of traditional spiral CT. Low-dose cone beam CT will reduce the density resolution of the image compared to traditional spiral CT images, and the scanning time of cone beam CT is long, which will introduce artifacts, such as motion artifacts. Some three-dimensional CT image segmentation software, such as Mimics software, uses a simple watershed segmentation algorithm, which has good results for traditional spiral CT images, but very poor results for low-dose cone beam CT. Currently, the CT image automatic segmentation software on the market is basically for traditional spiral CT images. Because the dose of traditional spiral CT images is high and the density resolution is relatively high, automatic segmentation is relatively easy. However, the image quality of low-dose cone beam CT is low, and it is very difficult for such products to achieve the segmentation effect of the present invention.

[0102] As Figures 3 - 6 shown, it is the segmentation effect of the knee joint level bones of a low-dose three-dimensional cone beam CT image by the Mimics software using a simple watershed segmentation algorithm. Figure 3 is the coronal plane of the fused image of the image and the bone segmentation result. The grayscale image is the three-dimensional image, and the yellow color is the segmentation result. Figure 3 The red straight line represents the cross-sectional position of the transverse plane, and the green straight line represents the cross-sectional position of the sagittal plane. Figure 4It is the cross-section of the fused image of the image and the bone segmentation result. The grayscale image is a three-dimensional image, and the yellow color represents the segmentation result. Figure 4 The green straight line of Figure 4 represents the section position of the sagittal plane, and the yellow straight line represents the section position of the coronal plane. Figure 5 It is the sagittal plane of the fused image of the image and the bone segmentation result. The grayscale image is a three-dimensional image, and the yellow color represents the segmentation result. Figure 5 The red straight line of Figure 5 represents the section position of the cross-section, and the yellow straight line represents the section position of the coronal plane. Figure 6 It is the three-dimensional rendering of the bone segmentation result. The segmented area (yellow part) does not contain complete bone tissue. A large amount of cancellous bone is misjudged as soft tissue, and many areas are not accurately segmented, resulting in a very poor segmentation effect.

[0103] As Figures 7 - 10 shown, it is the segmentation effect of the knee joint layer bone of the low-dose three-dimensional cone-beam CT image by the present invention. Figure 7 It is the coronal plane of the fused image of the image and the bone segmentation result. The grayscale image is a three-dimensional image. The red color represents the femoral segmentation result, the green color represents the tibial segmentation result, the blue color represents the fibular segmentation result, and the yellow color represents the talus segmentation result. Figure 8 It is the cross-section of the fused image of the image and the bone segmentation result. The grayscale image is a three-dimensional image. The red color represents the femoral segmentation result, and the cyan color represents the patella segmentation result. Figure 9 It is the sagittal plane of the fused image of the image and the bone segmentation result. The grayscale image is a three-dimensional image. The red color represents the femoral segmentation result, the green color represents the tibial segmentation result, and the cyan color represents the patella segmentation result. Figure 10 It is the three-dimensional rendering of the bone segmentation result. The red color represents the femoral segmentation result, the green color represents the tibial segmentation result, the blue color represents the fibular segmentation result, the yellow color represents the talus segmentation result, and the cyan color represents the patella segmentation result. It can be seen that the segmented area contains complete bone tissue (red part). By comparison, it can be known that the present invention can perform better segmentation on bone tissue, and different bones can be automatically classified and recognized.

[0104] It should be noted that if the image volume does not occupy much computer resources, the two-dimensional U-Net coarse segmentation can be not used, and the CT sequence of the three-dimensional lower limb bones can be directly sent into the three-dimensional U-Net for segmentation at one time or in groups.

[0105] (3) Extracting the isosurface: Assume that the threshold of the required isosurface is 0.5. Multiple cube voxels with a size of 2×2×2 are constructed with each pixel in the three-dimensional lower limb bone voxel model as the center. The pixel values at the vertices of the voxels are compared with the given threshold. First, find the voxels that intersect with the isosurface to obtain the first voxels; then, find the intersection points of the isosurface and the edges of the first voxels through interpolation, and connect the intersection points into triangles to form isosurface patches. The set of triangles in all the first voxels constitutes the three-dimensional lower limb bone network model.

[0106] (4) Extracting the surface data points.

[0107] A. Perform connectivity detection on the three-dimensional lower limb bone network model, classify the surface data points in the surface mesh model, and surface data points with direct or indirect connection paths between each other are classified into the same connected domain, and finally obtain multiple non-connected connected domains.

[0108] B. Calculate the volume of each connected domain. The volume calculation of each connected domain: Each pixel in the three-dimensional space is binary-classified according to whether it is inside the connected domain, and the number of all pixels inside the connected domain is counted, which is the volume of the connected domain. Set a volume threshold, and the connected domain with a volume greater than the volume threshold is identified as a valid connected domain, and the connected domain with a volume less than the volume threshold is identified as noise.

[0109] C. Calculate the central coordinates of each connected domain: Each pixel in the three-dimensional space is binary-classified according to whether it is inside the connected domain, and the average coordinate of all pixels inside the connected domain is counted, which is the central coordinate of the connected domain.

[0110] D. Classify the lower limb bones into the left femur, right femur, left tibia, right tibia, left talus, and right talus according to the central coordinates of the connected domains, and retain the surface data points of each lower limb bone.

[0111] In step S3, key points and mechanical axes can be extracted from the surface data points of the lower limb bones. The key points include the center point of the femoral head, the center point of the knee joint, and the center point of the ankle joint surface, and the mechanical axes include the femoral mechanical axis and the tibial mechanical axis.

[0112] (1) Center point of the femoral head

[0113] As an optional implementation manner, the extraction process of the center point of the femoral head is specifically as follows:

[0114] Select the data points of the proximal femur and the medial femur as the femoral head data points; Let the femoral head data point with the highest height be the femoral head vertex P1, with coordinates (x p1 , y p1 , z p1 ), let the femoral head data point closest to the inner side of the human body be the medial femoral head point P2, with coordinates (x p2 , y p2 , z p2 ), and the coordinates of the roughly estimated femoral head center o' are (x p1 , y p1 , z p2 );

[0115] Calculate the distances from o' to P1 and P2 respectively, and obtain the rough estimated radius r' of the femoral head after averaging; calculate the distances from the femoral head data points to o', and perform spherical fitting on the data points (x, y, z) whose distances are less than 1.2×r' to obtain the precise coordinates (x o ,y o ,z o ) of the center point o of the femoral head. The formula is as follows:

[0116] , where the horizontal line symbol above the parameter represents the averaging operation, which is a common mathematical symbol. For example represents taking the average of x, represents taking the average of the product result of xy, represents taking the average of x 2 for averaging. The interpretation principles of the remaining parameters are the same, and they will not be explained one by one in this embodiment.

[0117] As another alternative implementation manner, the process of extracting the center point of the femoral head is specifically as follows: Select multiple cross-sections at different positions or directions of the proximal femur, and respectively detect the center points of the two-dimensional circular structure of the femoral head within the cross-sections; and calculate the center of the sphere in three-dimensional space using multiple center points to obtain the center of the femoral head.

[0118] (2) Knee joint center point

[0119] As an alternative implementation manner, the process of extracting the knee joint center point is specifically as follows: Select the data points of the proximal tibia as the knee joint data points; set the knee joint data point closest to the inner side of the human body as the inner knee joint point P3, and set the knee joint data point closest to the outer side of the human body as the outer knee joint point P4; take the average of the coordinates of P3 and P4 to obtain the knee joint center point k.

[0120] As another alternative implementation manner, the process of extracting the knee joint center point is specifically as follows: Select the data points of the proximal tibia as the knee joint data points; take the average of the coordinates of all knee joint data points to obtain the knee joint center point.

[0121] (3) Center point of the ankle joint surface

[0122] As an alternative implementation manner, the process of extracting the center point of the ankle joint surface is specifically as follows: Select the data points of the proximal talus as the ankle joint data points; set the ankle joint data point closest to the inner side of the human body as the inner ankle joint point P5, and set the ankle joint data point closest to the outer side of the human body as the outer ankle joint point P6; take the average of the coordinates of P5 and P6 to obtain the ankle joint center point a.

[0123] As another alternative implementation manner, the center point of the ankle joint surface adopts the center points of the medial and lateral malleoli.

[0124] (4) Femoral mechanical axis

[0125] The line connecting the center point o of the femoral head and the center point k of the knee joint is the mechanical axis of the femur, and the direction of the mechanical axis of the femur is set from bottom to top; let the coordinates of the center of the femoral head be (x o , y o , z o ), the coordinates of the center point of the knee joint be (x k , y k , z k ), and the vector of the mechanical axis f of the femur is as shown in the following formula:

[0126] .

[0127] The unit vector of the mechanical axis of the femur is as shown in the following formula:

[0128] .

[0129] (5) Mechanical axis of the tibia

[0130] The line connecting the center point k of the knee joint and the center point of the ankle joint is the mechanical axis of the tibia, and the direction of the mechanical axis of the tibia is set from bottom to top; let the coordinates of the center point of the knee joint be (x k , y k , z k ), the coordinates of the center point of the ankle joint be (x a , y a , z a ), and the vector of the mechanical axis t of the femur is as shown in the following formula:

[0131] .

[0132] The unit vector of the mechanical axis of the femur is as shown in the following formula:

[0133] .

[0134] In step S4, the HKA angle can be an angle in two dimensions or three dimensions, and the HKA angle can also be visually displayed.

[0135] For example, if the HKA angle is an angle in two dimensions, the unit vectors of the mechanical axis of the femur and the mechanical axis of the tibia are both composed of the coordinates of the X-axis and the Z-axis.

[0136] The two-dimensional HKA angle refers to the angle Angle HKA 2D between the mechanical axis of the femur and the mechanical axis of the tibia in the coronal plane, and the calculation formula is as shown below:

[0137] .

[0138] For another example, if the HKA angle is an angle in three dimensions, the unit vectors of the femoral mechanical axis and the tibial mechanical axis are both composed of the coordinates of the X-axis, Y-axis, and Z-axis.

[0139] The three-dimensional HKA angle refers to the angle Angle between the femoral mechanical axis and the tibial mechanical axis in three-dimensional space HKA 3D , and the calculation formula is as follows:

[0140] .

[0141] As Figure 11 shown, the three-dimensional lower limb bone network model and the measurement results of the three-dimensional HKA angle are displayed simultaneously. The measurement results of the left lower limb are shown in green, and the measurement results of the right lower limb are shown in yellow. The varus angles of the two lower limb bones are 12.1° and 11.0° respectively.

[0142] Embodiment 2: A fully automatic measurement system for knee joint varus and valgus, which is used to implement a fully automatic measurement method for knee joint varus and valgus as described in Embodiment 1. As Figure 12 shown, it includes a data acquisition module, a surface extraction module, a center detection module, and an angle calculation module.

[0143] Among them, the data acquisition module is used to acquire three-dimensional lower limb bone data; the surface extraction module is used to segment and reconstruct the lower limb bones based on the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model, and extract the surface data points of the lower limb bones; the center detection module is used to extract key points including the femoral head center point, the knee joint center point, and the ankle joint surface center point from the surface data points of the lower limb bones, and determine the femoral mechanical axis and the tibial mechanical axis based on the key points; the angle calculation module is used to perform an inverse cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to calculate the HKA angle.

[0144] The present invention also records a computer terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a fully automatic measurement method for knee joint varus and valgus as described in Embodiment 1.

[0145] The present invention also records a computer-readable medium, on which a computer program is stored. When the computer program is executed by the processor, it can implement a fully automatic measurement method for knee joint varus and valgus as described in Embodiment 1.

[0146] Working principle: Based on prior knowledge, the present invention directly extracts key points from the segmentation results of lower limb bones, determines the femoral mechanical axis and the tibial mechanical axis according to the key points, and then performs an inverse cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to obtain the HKA angle. Its processing speed will not increase significantly with the increase of data volume, requires less computing power, takes less time, and has higher efficiency.

[0147] The present invention realizes the fully automated measurement of knee varus and valgus, solves the problem of high positioning requirements caused by two-dimensional projection in the previous X-ray detection method, solves the problem of manpower consumption caused by three-dimensional complex structures in the three-dimensional structure manual annotation method, and solves the problem of time-consuming caused by large data volume in the three-dimensional image data automatic detection method.

[0148] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing in the process Figure 1 one process or multiple processes and / or blocks Figure 1 a device for the functions specified in one block or multiple blocks.

[0150] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements in the process Figure 1 one process or multiple processes and / or blocks Figure 1 a device for the functions specified in one block or multiple blocks.

[0151] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the steps of the process Figure 1 in one process or a plurality of processes and / or boxes Figure 1 or steps for implementing the functions specified in one box or a plurality of boxes.

[0152] The specific embodiments described above further elaborate on the objectives, technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A fully automatic measurement method for genu varum and genu valgum, characterized in that, Including the following steps: Obtain three-dimensional lower limb bone data; After segmenting and reconstructing the lower limb bones based on the three-dimensional lower limb bone data, obtain a three-dimensional lower limb bone network model, and extract the surface data points of the lower limb bones; Extract key points including the center point of the femoral head, the center point of the knee joint, and the center point of the ankle joint surface from the surface data points of the lower limb bones, and determine the femoral mechanical axis and the tibial mechanical axis based on the key points; Perform inverse cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis, and calculate the HKA angle; The segmentation of the lower limb bones is realized by the method of deep learning segmentation, including: Train the deep learning segmentation network for three-dimensional lower limb bones; Perform binary classification on the pixels in the three-dimensional space by whether they are inside the lower limb bone region through the deep learning segmentation network; if the pixel is in the non-lower limb bone region, assign a value of 0; if the pixel is in the lower limb bone region, assign a value of 1; Take the classification result of the binary classification as the three-dimensional lower limb bone voxel model; The training of the deep learning segmentation network for three-dimensional lower limb bones includes: Perform a frontal projection in the coronal plane on the full lower limb sequence composed of multiple tomographic images to form a two-dimensional orthographic projection image; Use a two-dimensional U-Net to perform rough segmentation on the two-dimensional orthographic projection image to roughly locate each bone; Put the tomographic images into a three-dimensional U-Net in groups of multiple layers of images for fine bone segmentation, and refer to the results of the rough positioning of each bone during segmentation.

2. The full-automatic measurement method for knee varus and valgus according to claim 1, wherein This method further includes: There are overlapping tomographic images between adjacent groups of images; After the segmentation of adjacent groups of images is completed, perform intersection fusion on the segmentation results of the overlapping tomographic images; if the bone voxel appears in the overlapping area of both groups, the bone voxel is retained, otherwise it is deleted.

3. The full-automatic measurement method for knee varus and valgus according to claim 1, characterized in that, The reconstruction of the lower limb bones is realized by the method of extracting the isosurface or registering the deficit model.

4. The full-automatic measurement method for knee varus and valgus according to claim 3, characterized in that The reconstruction of the lower limb bones is realized by the method of extracting the isosurface, including: Set the threshold of the isosurface to be M; Construct multiple cube voxels of size N×N×N with each pixel in the three-dimensional lower limb bone voxel model obtained after segmentation as the center; Compare the pixel values at the vertices of the cube voxel with the threshold M, and screen out the first voxels that intersect with the isosurface; Find the intersection points of the isosurface and the edges of the first voxels by interpolation, and connect the intersection points into triangles to form an isosurface patch. The three-dimensional lower limb bone network model is composed of the set of triangles in all the first voxels.

5. A fully automatic measurement method for knee varus and valgus according to claim 1, characterized in that, The extraction of the surface data points of the lower limb bones includes: Perform connectivity detection on the three-dimensional lower limb bone network model to obtain multiple non-connected connected domains; Calculate the center coordinates of each connected domain; Classify the lower limb bones into the left femur, right femur, left tibia, right tibia, left talus, and right talus according to the center coordinates of the connected domains, and retain the surface data points of each lower limb bone.

6. A fully automatic measurement method for knee varus and valgus according to claim 5, characterized in that, The calculation of the center coordinates of each connected domain includes: Perform binary classification on each pixel in the three-dimensional space by whether it is inside the connected domain, and statistically calculate the average value of the coordinates of all pixels inside the connected domain to obtain the center coordinates of the connected domain.

7. A fully automatic measurement method for knee varus and valgus according to claim 5, characterized in that This method further includes: Set a volume threshold; Calculate the volume of each connected component, where the volume of the connected component is the sum of the number of pixels inside the connected component; If the volume of the connected component is greater than the volume threshold, the corresponding connected component is identified as a valid connected component; and if the volume of the connected component is less than the volume threshold, the corresponding connected component is identified as noise.

8. A fully automatic measurement method for knee joint varus and valgus according to claim 1, characterized in that, The process of extracting the center point of the femoral head is specifically as follows: Select data points of the proximal femur and the medial femur as femoral head data points; Select the femoral head data point with the highest height as the femoral head vertex and the femoral head data point closest to the medial side of the human body as the medial femoral head point, and combine the Z-axis coordinate of the medial femoral head point and the X-axis and Y-axis coordinates in the femoral head vertex to determine the initial femoral head center; Calculate the first distances from the initial femoral head center to the femoral head vertex and the medial femoral head point respectively, and use the average value of the two calculated first distances as the rough estimated radius of the femoral head; Calculate the second distance from the femoral head data point to the initial femoral head center, and perform spherical fitting on the femoral head data points with the second distance less than 1.2 times the rough estimated radius to obtain the accurately determined center point of the femoral head.

9. A fully automatic measurement method for knee varus and valgus according to claim 1, characterized in that, The process of extracting the center point of the femoral head is specifically as follows: Select cross-sections at multiple different positions or directions of the proximal femur, and respectively detect the center of the two-dimensional circular femoral head structure within the cross-sections; And calculate the center of the sphere in three-dimensional space using multiple centers to obtain the center of the femoral head.

10. The fully automatic measurement method for knee varus and valgus according to claim 1, characterized in that, The process of extracting the center point of the knee joint is specifically as follows: Select data points of the proximal tibia as knee joint data points; Select the knee joint data point closest to the medial side of the human body as the medial knee joint point and the knee joint data point closest to the lateral side of the human body as the lateral knee joint point; Average the coordinates of the medial knee joint point and the lateral knee joint point to obtain the center point of the knee joint.

11. A fully automatic measurement method for knee varus and valgus according to claim 1, characterized in that, The process of extracting the center point of the knee joint is specifically as follows: Select data points of the proximal tibia as knee joint data points; Average the coordinates of all the knee joint data points to obtain the center point of the knee joint.

12. The full-automatic measurement method for genu varum and genu valgum according to claim 1, wherein The process of extracting the center point of the ankle joint surface is specifically as follows: Select data points of the proximal talus as ankle joint data points; Select the ankle joint data point closest to the medial side of the human body as the medial ankle joint point and select the ankle joint data point closest to the lateral side of the human body as the lateral ankle joint point; Average the coordinates of the medial ankle joint point and the lateral ankle joint point to obtain the center point of the ankle joint.

13. The full-automatic measurement method for knee varus and valgus according to claim 1, wherein, The center point of the ankle joint surface adopts the center points of the medial and lateral malleoli.

14. The fully automatic measurement method for knee varus and valgus according to claim 1, characterized in that, If the HKA angle is an angle in two dimensions, the unit vectors of the femoral mechanical axis and the tibial mechanical axis are both composed of the coordinates of the X-axis and the Z-axis; And / or, if the HKA angle is an angle in three dimensions, the unit vectors of the femoral mechanical axis and the tibial mechanical axis are both composed of the coordinates of the X-axis, the Y-axis, and the Z-axis.

15. A fully automatic measurement system for genu varum and genu valgum, characterized in that, This system is used to implement a fully automatic measurement method for knee varus and valgus as described in any one of claims 1-14, including: A data acquisition module for acquiring three-dimensional lower limb bone data; A surface extraction module, which is used to segment and reconstruct the lower limb bones based on the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model, and extract the surface data points of the lower limb bones; A center detection module, which is used to extract key points including the center point of the femoral head, the center point of the knee joint, and the center point of the ankle joint surface from the surface data points of the lower limb bones, and determine the femoral mechanical axis and the tibial mechanical axis based on the key points; An angle calculation module, which is used to perform an inverse cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to calculate the HKA angle; The segmentation of the lower limb bones is realized by means of deep learning segmentation, including: Training a deep learning segmentation network for three-dimensional lower limb bones; Performing binary classification on the pixels in the three-dimensional space by the deep learning segmentation network according to whether they are inside the lower limb bone region; if the pixel is in the non-lower limb bone region, it is assigned a value of 0; if the pixel is in the lower limb bone region, it is assigned a value of 1; Taking the classification result of the binary classification as a three-dimensional lower limb bone voxel model; The training of the deep learning segmentation network for three-dimensional lower limb bones includes: Performing a coronal forward projection on the full lower limb sequence composed of multiple tomographic images to form a two-dimensional forward projection image; Using a two-dimensional U-Net to perform rough segmentation on the two-dimensional forward projection image to roughly locate each bone; Putting the tomographic images into a three-dimensional U-Net in groups of multiple layers of images for fine bone segmentation, and referring to the results of the rough positioning of each bone during segmentation.

16. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it realizes a fully automatic measurement method for knee varus and valgus as described in any one of claims 1-14.

17. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it can realize a fully automatic measurement method for knee varus and valgus as described in any one of claims 1-14.

Images