Full-automatic measurement method and system for knee joint varus and valgus, terminal and medium

Abstract

The invention discloses a full-automatic measurement method and system for knee joint varus and valgus, a terminal and a medium, and relates to the technical field of image processing, and the key points of the technical scheme are as follows: obtaining three-dimensional lower limb bone data; segmenting and reconstructing lower limb bones according to the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model, and extracting lower limb bone surface data points; key points including a femoral head center point, a knee joint center point and an ankle joint surface center point are extracted from the lower limb bone surface data points, and a femoral mechanical axis and a tibia mechanical axis are determined according to the key points; and performing arc cosine solution on the dot product between the unit vector of the thighbone mechanical axis and the unit vector of the tibia mechanical axis, and calculating to obtain an HKA angle. According to the method, the key points are directly extracted from the segmentation result of the lower limb bone based on the priori knowledge, the processing speed is not greatly increased along with the increase of the data volume, the requirement on the computing power is lower, the consumed time is less, and the efficiency is higher.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a fully automatic measurement method, system, terminal and medium for knee valgus and valgus. Background Art

[0002] The hip-knee-ankle (HKA) angle refers to the angle between the mechanical axis of the femur and the mechanical axis of the tibia, and is an important indicator for evaluating valgus and valgus deformity of the knee joint. The mechanical axis of the femur refers to the line from the center of the femoral head to the center of the knee joint, and the mechanical axis of the tibia refers to the line from the center of the knee joint to the center of the ankle joint.

[0003] The existing fully automatic measurement methods for knee valgus and varus mainly include X-ray detection methods and three-dimensional image detection methods based on deep learning. In the X-ray detection method, deep learning networks or traditional methods are generally used to segment the lower limbs in the X-ray or detect key points; since the X-ray is a two-dimensional projection imaging, its important quality standard is that the patella is centered between the femoral condyles, so the patient's standing position and leg position have a greater impact on the measurement results of knee valgus and varus, and it is easy to cause measurement errors due to factors such as the patient himself or the technician's operation. In the three-dimensional image detection method based on deep learning, a deep learning network is generally used to predict the key points of the lower limbs in the three-dimensional image; since the amount of three-dimensional image data is much larger than that of two-dimensional image data, in order to detect multiple key points in the global image, the deep learning network needs to perform a large amount of calculations, which is time-consuming. Summary of the invention

[0004] In order to address the deficiencies in the prior art, the purpose of the present invention is to provide a fully automatic measurement method, system, terminal and medium for knee valgus and valgus. Based on prior knowledge, key points are directly extracted from the segmentation results of the lower limb bones, and the femoral mechanical axis and the tibial mechanical axis are determined according to the key points. The dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis is then solved by arc cosine to obtain the HKA angle. The processing speed will not increase significantly with the increase of data volume, the computing power requirement is smaller, the time consumption is less, and the efficiency is higher.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions: A first aspect of the present invention provides a fully automatic method for measuring knee valgus and valgus, comprising the following steps: Acquire three-dimensional lower limb bone data; Segmenting and reconstructing the lower limb bones according to the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model, and extracting lower limb bone surface data points; Extracting key points including the center point of the femoral head, the center point of the knee joint and the center point of the ankle joint surface from the lower limb bone surface data points, and determining the femoral mechanical axis and the tibial mechanical axis based on the key points; The HKA angle is calculated by performing an arc cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis.

[0006] In one implementation, the segmentation of the lower limb bones is achieved by threshold segmentation or deep learning segmentation.

[0007] In one implementation, the lower limb bones are segmented by a deep learning segmentation method, including: Training a deep learning segmentation network for 3D lower limb bones; The deep learning segmentation network is used to classify pixels in the three-dimensional space into two categories according to whether they are in the lower limb bone region; if the pixel is in the non-lower limb bone region, the value is assigned to 0; if the pixel is in the lower limb bone region, the value is assigned to 1; The binary classification results are used as the three-dimensional lower limb bone voxel model.

[0008] In one implementation, the deep learning segmentation network for training three-dimensional lower limb bones includes: The whole lower limb sequence consisting of multiple tomographic images was forward projected in the coronal plane to form a two-dimensional forward projection image; Use a two-dimensional U-Net to roughly segment the two-dimensional orthographic projection image and roughly locate each bone; The tomographic image is put into a three-dimensional U-Net with multiple layers of images as a group of images for fine bone segmentation, and the results of the rough positioning of each bone are referred to during the segmentation.

[0009] In one implementation, the method further includes: Overlapping tomograms are provided between adjacent group images; After the segmentation of the adjacent group images is completed, the segmentation results of the overlapping tomograms are intersected and fused; if the bone voxel appears in the overlapping area of ​​the two groups at the same time, the bone voxel is retained, otherwise it is deleted.

[0010] In one implementation, the reconstruction of the lower limb bones is achieved by extracting isosurfaces or scarce model registration.

[0011] In one implementation, the lower limb bones are reconstructed by extracting isosurfaces, including: Let the threshold of the desired isosurface be M; Construct multiple N×N×N cube voxels with each pixel in the three-dimensional lower limb bone voxel model obtained after segmentation as the center; Compare the pixel values ​​at each vertex of the cube voxel with a threshold M, and select the first voxel intersecting with the isosurface; The intersection points of the isosurface and the edges of the first volume element are obtained by interpolation, and the intersection points are connected into triangles to form an isosurface patch. The three-dimensional lower limb bone network model is formed by the collection of triangles in all the first volume elements.

[0012] In one implementation, extracting lower limb bone surface data points includes: Performing connectivity detection on the three-dimensional lower limb bone network model to obtain a plurality of interconnected domains that are not connected to each other; Calculate the center coordinates of each of the connected domains; The lower limb bones are classified into left femur, right femur, left tibia, right tibia, left talus and right talus according to the central coordinates of the connected domain, and the surface data points of each lower limb bone are retained.

[0013] In one implementation, the calculating the center coordinates of each of the connected regions includes: Each pixel in the three-dimensional space is classified into two categories according to whether it is inside the connected domain, and the average coordinates of all pixels inside the connected domain are counted to obtain the central coordinates of the connected domain.

[0014] In one implementation, the method further includes: Set volume thresholds; Calculating the volume of each connected domain, where the volume of the connected domain is the sum of the number of pixels inside the connected domain; If the volume of the connected domain is greater than the volume threshold, the corresponding connected domain is identified as a valid connected domain; and if the volume of the connected domain is less than the volume threshold, the corresponding connected domain is identified as noise.

[0015] In one implementation, the process of extracting the center point of the femoral head is specifically as follows: The data points of the proximal femur and the medial femur are selected as the femoral head data points; The femoral head data point with the highest height is selected as the femoral head apex and the femoral head data point closest to the inner side of the human body is selected as the medial point of the femoral head, and the initial femoral head center is determined by combining the Z-axis coordinate of the medial point of the femoral head and the X-axis coordinate and the Y-axis coordinate of the femoral head apex; Calculating first distances from the initial femoral head center to the femoral head apex and the medial point of the femoral head respectively, and taking the average of the two calculated first distances as a rough estimated radius of the femoral head; The second distance from the femoral head data point to the initial femoral head center is calculated, and spherical fitting is performed on the femoral head data points whose second distance is less than 1.2 times the rough estimated radius to obtain the accurately determined femoral head center point.

[0016] In one implementation, the process of extracting the center point of the femoral head is specifically as follows: Select multiple sections at different positions or directions of the proximal femur, and perform center detection on the two-dimensional circular structure of the femoral head in the sections respectively; And, the center of a sphere in three-dimensional space is calculated using multiple circle centers to obtain the femoral head center.

[0017] In one implementation, the process of extracting the center point of the knee joint is specifically as follows: Select the data points at the proximal end of the tibia as the knee joint data points; Selecting the knee joint data point closest to the inner side of the human body as the inner side of the knee joint point and the knee joint data point closest to the outer side of the human body as the outer side of the knee joint point; The coordinates of the medial point of the knee joint and the lateral point of the knee joint are averaged to obtain the center point of the knee joint.

[0018] In one implementation, the process of extracting the center point of the knee joint is specifically as follows: Select the data points at the proximal end of the tibia as the knee joint data points; The coordinates of all the knee joint data points are averaged to obtain the center point of the knee joint.

[0019] In one implementation, the process of extracting the center point of the ankle joint surface is specifically as follows: The data points at the proximal end of the talus were selected as the ankle joint data points; The ankle joint data point closest to the inside of the human body is selected as the inner ankle joint point, and the ankle joint data point closest to the outside of the human body is selected as the outer ankle joint point; The coordinates of the medial point of the ankle joint and the lateral point of the ankle joint are averaged to obtain the center point of the ankle joint.

[0020] In one implementation, the center point of the ankle joint surface is the center point of the medial and lateral malleolus.

[0021] In one implementation, if the HKA angle is a two-dimensional angle, the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis are both composed of coordinates of the X-axis and the Z-axis; And / or, if the HKA angle is a three-dimensional angle, the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis are both composed of coordinates of the X-axis, the Y-axis, and the Z-axis.

[0022] A second aspect of the present invention provides a fully automatic measurement system for knee valgus and valgus, the system being used to implement a fully automatic measurement method for knee valgus and valgus as provided in the first aspect of the present invention, comprising: A data acquisition module, used for acquiring three-dimensional lower limb bone data; A surface extraction module, used to segment and reconstruct the lower limb bones according to the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model, and extract lower limb bone surface data points; A center detection module, used to extract key points including the center point of the femoral head, the center point of the knee joint and the center point of the ankle joint surface from the lower limb bone surface data points, and determine the femoral mechanical axis and the tibial mechanical axis based on the key points; The angle calculation module is used to perform arc cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to calculate the HKA angle.

[0023] The third aspect of the present invention provides a computer terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a fully automatic measurement method for knee valgus and valgus as provided in the first aspect of the present invention.

[0024] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement a fully automatic measurement method for knee valgus and valgus as provided in the first aspect of the present invention.

[0025] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention provides a fully automatic measurement method for knee valgus and valgus. The method directly extracts key points from the segmentation results of the lower limb bones based on prior knowledge, and determines the femoral mechanical axis and the tibial mechanical axis according to the key points. The dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis is then solved by arc cosine to obtain the HKA angle. The processing speed will not increase significantly with the increase of data volume, the computing power requirement is smaller, the time consumption is less, and the efficiency is higher. 2. The deep learning model described in the present invention can be used not only for traditional spiral CT data, but also for low-dose cone-beam CT data; the dose of low-dose cone-beam CT used by the deep learning model in the present invention is equivalent to 10% to 20% of the dose of traditional spiral CT, and the present invention can achieve a good segmentation effect on bone tissue, and different bones can be automatically classified and identified; 3. The present invention can not only realize HKA angle calculation in three dimensions, but also be applicable to HKA angle calculation in two dimensions, and its application flexibility is relatively strong; 4. The present invention realizes fully automated measurement of knee valgus and valgus, solves the problem of high positioning requirements caused by two-dimensional projection in previous X-ray detection methods, solves the problem of manpower consumption and measurement errors caused by three-dimensional complex structures in manual three-dimensional structure annotation methods, and solves the time-consuming problem caused by large amounts of data in automatic detection methods of three-dimensional image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings: Figure 1 is a flow chart of Embodiment 1 of the present invention; Figure 2 is a flow chart of surface extraction in Example 1 of the present invention; Figure 3 It is the coronal plane of the fusion image of the image and the bone segmentation result in the prior art; Figure 4 It is a cross section of the fusion graph of the image and the bone segmentation result in the prior art; Figure 5 It is the sagittal plane of the fusion image of the image and the bone segmentation result in the prior art; Figure 6 It is a three-dimensional rendering of the bone segmentation result in the prior art; Figure 7 is the coronal plane of the fusion image of the image and the bone segmentation result in Example 1 of the present invention; Figure 8 is a cross section of the fusion image of the image and the bone segmentation result in Example 1 of the present invention; Fig. 9 is the sagittal plane of the fusion image of the image and the bone segmentation result in Example 1 of the present invention; Fig.10 It is a three-dimensional rendering of the skeleton segmentation result in Example 1 of the present invention; Fig.11 is a visualized schematic diagram of the angle between the lower limb bones and HKA in Example 1 of the present invention; Fig.12 It is a system block diagram in Example 2 of the present invention. DETAILED DESCRIPTION

[0027] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0028] Embodiment 1: A fully automatic measurement method for knee valgus and valgus, such as Figure 1 As shown, the following steps are included: S1: Acquire three-dimensional lower limb bone data; S2: Segment and reconstruct the lower limb bones based on the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model, and extract the lower limb bone surface data points; S3: Extract key points including the center point of the femoral head, the center point of the knee joint and the center point of the ankle joint surface from the data points on the lower limb bone surface, and determine the femoral mechanical axis and the tibial mechanical axis based on the key points; S4: The HKA angle is calculated by solving the arc cosine of the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis.

[0029] In step S1, three-dimensional lower limb bone data may be separated from three-dimensional data, and the three-dimensional data may be composed of voxels, point clouds or meshes, and the voxels are such as CT (computed tomography) scan data and MRI (magnetic resonance imaging) data.

[0030] In step S2, the segmentation of the lower limb bones can be achieved by a threshold segmentation method or a deep learning segmentation method.

[0031] In addition, reconstruction of lower limb bones can be achieved by extracting isosurfaces or by scarce model registration.

[0032] Take the 3D data of human body scanned by CT as an example. Figure 2 As shown, it is specifically implemented by the following steps.

[0033] (1) Input CT scan 3D data of the human body.

[0034] (2) Deep learning segmentation: A deep learning segmentation network is trained for the three-dimensional lower limb bones. Each pixel in the three-dimensional space is classified into two categories according to whether it is inside the lower limb bone area. The values ​​are assigned as 0 or 1, where 0 represents the non-lower limb bone area and 1 represents the lower limb bone area. The classification result is the three-dimensional lower limb bone voxel model.

[0035] The three-dimensional CT sequence of lower limb bones generally uses a layer interval of 1.0mm and a layer thickness of 2.0mm. The tomogram is a 512×512 matrix. For patients of average height, there are about 1,000 lower limb images, and the storage capacity is about 1GB. Directly sending them to the AI ​​model for training or derivation at one time will take up a lot of computer (memory and GPU video memory) resources. Directly sending the entire case into the AI ​​model will increase the cost of development and use. Inputting tomographic images into the AI ​​model in batches will cause the AI ​​model to be unable to obtain global information. Therefore, the present invention adopts a two-stage AI model architecture and training method.

[0036] First, the whole lower limb sequence of the patient (about 1000 tomograms, equivalent to 1000mm spatial length of the Z axis) is used for coronal forward projection to form a two-dimensional forward projection image. First, the two-dimensional U-Net is used to roughly segment the two-dimensional forward projection image and roughly locate each bone. Then, the tomographic images are put into the three-dimensional U-Net as a group of 32 layers for fine bone segmentation. During segmentation, the bone positioning information of the two-dimensional U-Net will be referred to. This global and local segmentation method can not only avoid the large occupation of computer resources, but also allow the model to fully consider global information and prevent the occurrence of mis-segmentation and misclassification problems. The model obtained by the preliminary segmentation of the three-dimensional lower limb bone data is the three-dimensional lower limb bone voxel model, and then the three-dimensional lower limb bone network model can be obtained by reconstruction based on the three-dimensional lower limb bone voxel model.

[0037] In addition, in order to avoid misalignment when the segmentation results of adjacent groups are stitched together, the groups are overlapped, generally 1 / 4 of the number of images in a group. For example, if a group has 32 images, then 8 images of the current group and the previous group overlap, and 8 images of the current group and the next group overlap. After the adjacent groups are segmented, the segmentation results of the overlapping parts are intersected and fused, that is, the identified bone voxels appear in the overlapping areas of the two groups at the same time, and they are retained, otherwise they are deleted.

[0038] The present invention can be used not only for traditional spiral CT data, but also for low-dose cone-beam CT data. The dose of low-dose cone-beam CT used by the deep learning model of the present invention is equivalent to 10% to 20% of the dose of traditional spiral CT. Low-dose cone-beam CT will reduce the density resolution of the image relative to traditional spiral CT images, and the long scanning time of cone-beam CT will introduce artifacts, such as motion artifacts. Some three-dimensional CT image segmentation software, such as Mimics software, uses a simple watershed segmentation algorithm, which has a good effect on traditional spiral CT images, but has a poor effect on low-dose cone-beam CT. The automatic segmentation software for CT images currently on the market is basically for traditional spiral CT images. Because traditional spiral CT images have high doses and high density resolution, automatic segmentation is relatively easy. However, the image quality of low-dose cone-beam CT is low, and it is difficult for such products to achieve the segmentation effect of the present invention.

[0039] like Figure 3-Figure 6 As shown, it shows the effect of bone segmentation at the knee joint level of low-dose three-dimensional cone-beam CT images using a simple watershed segmentation algorithm using Mimics software. Figure 3 It is the coronal plane of the fusion image of the image and the bone segmentation result. The grayscale image is the three-dimensional image, and the yellow image is the segmentation result. Figure 3 The red straight line represents the position of the cross-sectional section, and the green straight line represents the position of the sagittal section. Figure 4It is the cross section of the fusion image of the image and the bone segmentation result. The grayscale image is the three-dimensional image, and the yellow image is the segmentation result. Figure 4 The green straight line represents the section position of the sagittal plane, and the yellow straight line represents the section position of the coronal plane. Figure 5 It is the sagittal plane of the fusion image of the image and the bone segmentation result. The grayscale image is the three-dimensional image, and the yellow image is the segmentation result. Figure 5 The red straight line represents the position of the cross-sectional section, and the yellow straight line represents the position of the coronal section. Figure 6 This is a 3D rendering of the bone segmentation result. The segmented area (yellow part) does not contain complete bone tissue. A large amount of cancellous bone is misjudged as soft tissue. Many areas are not accurately segmented, and the segmentation effect is very poor.

[0040] like Figure 7-Figure 10 As shown, it is the segmentation effect of the present invention on the bones at the knee joint level of low-dose three-dimensional cone-beam CT images. Figure 7 It is the coronal plane of the fusion image of the image and bone segmentation results. The grayscale image is a three-dimensional image. The red image is the femur segmentation result, the green image is the tibia segmentation result, the blue image is the fibula segmentation result, and the yellow image is the talus segmentation result. Figure 8 It is the cross section of the fusion image of the image and the bone segmentation result. The grayscale image is a three-dimensional image. The red image is the femur segmentation result, and the cyan image is the patella segmentation result. Fig. 9 It is the sagittal plane of the fusion image of the image and the bone segmentation result. The grayscale image is a three-dimensional image. The red image is the femur segmentation result, the green image is the tibia segmentation result, and the cyan image is the patella segmentation result. Fig.10 This is a 3D rendering of the bone segmentation result. The red one is the femur segmentation result, the green one is the tibia segmentation result, the blue one is the fibula segmentation result, the yellow one is the talus segmentation result, and the cyan one is the patella segmentation result. It can be seen that the segmented area contains the complete bone tissue (the red part). By comparison, it can be seen that the present invention can perform better segmentation of bone tissue, and different bones can be automatically classified and identified.

[0041] It should be noted that if the image volume is not large relative to the computer resources, the two-dimensional U-Net coarse segmentation can be omitted, and the three-dimensional lower limb bone CT series can be directly sent to the three-dimensional U-Net segmentation at one time or in groups.

[0042] (3) Extracting isosurfaces: Assume that the threshold of the isosurface is 0.5, and construct multiple 2×2×2 cubic voxels with each pixel in the three-dimensional lower limb bone voxel model as the center. Compare the pixel value at each vertex of the voxel with the given threshold. First, find the voxel that intersects with the isosurface to obtain the first voxel. Then, interpolate the intersection of the isosurface and the edge of the first voxel to obtain the isosurface patch, and connect the intersections into triangles to form an isosurface patch. The set of triangles in all the first voxels constitutes the three-dimensional lower limb bone network model.

[0043] (4) Extract surface data points.

[0044] A. Perform connectivity detection on the three-dimensional lower limb bone network model, classify the surface data points in the surface mesh model, and classify the surface data points that have direct or indirect connection paths to each other into the same connected domain, and finally obtain multiple connected domains that are not connected to each other.

[0045] B. Calculate the volume of each connected domain: Classify each pixel in the three-dimensional space into two categories according to whether it is inside the connected domain, and count the number of pixels inside the connected domain, which is the volume of the connected domain. Set a volume threshold, and identify the connected domain with a volume greater than the volume threshold as a valid connected domain, and the connected domain with a volume less than the volume threshold as noise.

[0046] C. Calculate the center coordinates of each connected domain: Classify each pixel in the three-dimensional space into two categories according to whether it is inside the connected domain, and calculate the average coordinate value of all pixels inside the connected domain, which is the center coordinate of the connected domain.

[0047] D. Classify the lower limb bones into left femur, right femur, left tibia, right tibia, left talus and right talus according to the central coordinates of the connected domain, and retain the surface data points of each lower limb bone.

[0048] In step S3, key points and mechanical axes can be extracted from the lower limb bone surface data points, wherein the key points include the center point of the femoral head, the center point of the knee joint, and the center point of the ankle joint surface, and the mechanical axes include the femoral mechanical axis and the tibia mechanical axis.

[0049] (1) Femoral head center point As an optional implementation, the process of extracting the center point of the femoral head is specifically as follows: Select the data points of the proximal femur and the medial femur as the femoral head data points; set the highest femoral head data point as the femoral head vertex P1, with coordinates (x p1 ,y p1 , z p1 ), let the femoral head data point closest to the inner side of the human body be the inner side point P2 of the femoral head, and its coordinates are (x p2 ,y p2 , z p2 ), according to the rough estimate of the coordinates of the femoral head center o' is (x p1 ,y p1 , z p2 ); The distances from o' to P1 and P2 were calculated respectively, and the rough radius r' of the femoral head was obtained by averaging. The distance from the femoral head data point to o' was calculated, and spherical fitting was performed on the data points (x, y, z) with a distance less than 1.2 × r' to obtain the precise coordinates of the femoral head center point o (x o ,y o, z o ), the formula is as follows: , where the horizontal line symbol above the parameter represents the averaging operation, which is a common mathematical symbol, such as It means to find the average value of x. It means to find the average value of the product of xy. Indicates x 2 The average value is calculated, and the interpretation of the remaining parameters is the same principle, which will not be explained one by one in this embodiment.

[0050] As another optional implementation, the process of extracting the center point of the femoral head is specifically as follows: selecting multiple sections at different positions or directions of the proximal femur, and performing center detection on the two-dimensional circular structure of the femoral head in the sections respectively; and using multiple centers of circles to calculate the center of the sphere in three-dimensional space to obtain the center of the femoral head.

[0051] (2) Knee joint center As an optional implementation, the extraction process of the knee joint center point is specifically as follows: select the data point of the proximal tibia as the knee joint data point; set the knee joint data point closest to the inner side of the human body as the inner knee joint point P3, and set the knee joint data point closest to the outer side of the human body as the outer knee joint point P4; average the coordinates of P3 and P4 to obtain the knee joint center point k.

[0052] As another optional implementation, the process of extracting the center point of the knee joint is specifically as follows: selecting the data point of the proximal end of the tibia as the knee joint data point; averaging the coordinates of all the knee joint data points to obtain the center point of the knee joint.

[0053] (3) Ankle joint center point As an optional implementation, the process of extracting the center point of the ankle joint surface is specifically as follows: select the data point at the proximal end of the talus as the ankle joint data point; set the ankle joint data point closest to the inner side of the human body as the medial ankle joint point P5, and set the ankle joint data point closest to the outer side of the human body as the lateral ankle joint point P6; average the coordinates of P5 and P6 to obtain the ankle joint center point a.

[0054] As another optional embodiment, the center point of the ankle joint surface adopts the center point of the medial and lateral malleolus.

[0055] (4) Femoral mechanical axis The line connecting the center point o of the femoral head and the center point k of the knee joint is the mechanical axis of the femur. The direction of the mechanical axis of the femur is set from bottom to top. The coordinates of the center of the femoral head are (x o ,y o , z o ), the coordinates of the center point of the knee joint are (x k ,y k , z k), the vector of the femoral mechanical axis f As shown in the following formula: .

[0056] Unit vector of the femoral mechanical axis As shown in the following formula: .

[0057] (5) Tibial mechanical axis The line connecting the center point k of the knee joint and the center point of the ankle joint is the tibial mechanical axis. The direction of the tibial mechanical axis is set from bottom to top. The coordinates of the center point of the knee joint are (x k ,y k , z k ), the coordinates of the center point of the ankle joint are (x a ,y a , z a ), the vector of the femoral mechanical axis t As shown in the following formula: .

[0058] Unit vector of the femoral mechanical axis As shown in the following formula: .

[0059] In step S4, the HKA angle may be a two-dimensional angle or a three-dimensional angle, and the HKA angle may also be displayed visually.

[0060] For example, if the HKA angle is an angle in two dimensions, the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis are both composed of the coordinates of the X-axis and the Z-axis.

[0061] The 2D HKA angle refers to the angle between the mechanical axis of the femur and the mechanical axis of the tibia on the coronal plane. HKA 2D , the calculation formula is as follows: .

[0062] For another example, if the HKA angle is a three-dimensional angle, the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis are both composed of the coordinates of the X-axis, the Y-axis, and the Z-axis.

[0063] The 3D HKA angle refers to the angle between the femoral mechanical axis and the tibial mechanical axis in 3D space. HKA 3D , the calculation formula is as follows: .

[0064] like Fig.11As shown, the three-dimensional lower limb bone network model and the three-dimensional HKA angle measurement results are displayed simultaneously, with the left lower limb measurement results displayed in green and the right lower limb measurement results displayed in yellow. The varus angles of the two lower limb bones are 12.1° and 11.0° respectively.

[0065] Embodiment 2: A fully automatic measurement system for knee valgus and valgus, the system is used to implement a fully automatic measurement method for knee valgus and valgus as described in Embodiment 1, Fig.12 As shown, it includes a data acquisition module, a surface extraction module, a center detection module and an angle calculation module.

[0066] Among them, the data acquisition module is used to obtain three-dimensional lower limb bone data; the surface extraction module is used to segment and reconstruct the lower limb bones according to the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model, and extract the lower limb bone surface data points; the center detection module is used to extract key points including the center point of the femoral head, the center point of the knee joint and the center point of the ankle joint surface from the lower limb bone surface data points, and determine the femoral mechanical axis and the tibial mechanical axis based on the key points; the angle calculation module is used to perform arccosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to calculate the HKA angle.

[0067] The present invention also describes a computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a fully automatic measurement method for knee valgus and valgus as described in Example 1 is implemented.

[0068] The present invention also describes a computer-readable medium on which a computer program is stored. The computer program is executed by a processor to implement a fully automatic measurement method for knee valgus and valgus as described in Example 1.

[0069] Working principle: The present invention directly extracts key points from the segmentation results of the lower limb bones based on prior knowledge, and determines the femoral mechanical axis and the tibial mechanical axis based on the key points, and then performs arc cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to obtain the HKA angle. The processing speed will not increase significantly with the increase of data volume, the computing power requirements are smaller, the time consumption is less, and the efficiency is higher.

[0070] The present invention realizes fully automated measurement of knee valgus and valgus, solves the problem of high positioning requirements caused by two-dimensional projection in previous X-ray film detection methods, solves the problem of manpower consumption caused by three-dimensional complex structures in manual three-dimensional structure annotation methods, and solves the time-consuming problem caused by large data volume in automatic three-dimensional image data detection methods.

[0071] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0072] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0075] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A fully automatic method for measuring knee valgus and valgus, characterized in that: The following steps are involved: Acquire three-dimensional lower limb bone data; Segmenting and reconstructing the lower limb bones according to the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model, and extracting lower limb bone surface data points; Extracting key points including the center point of the femoral head, the center point of the knee joint and the center point of the ankle joint surface from the lower limb bone surface data points, and determining the femoral mechanical axis and the tibial mechanical axis based on the key points; The HKA angle is calculated by performing an arc cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis.

2. A fully automatic measurement method for knee valgus and valgus according to claim 1, characterized in that: The segmentation of lower limb bones is achieved through threshold segmentation or deep learning segmentation methods.

3. A fully automatic measurement method for knee valgus and valgus according to claim 2, characterized in that: The segmentation of lower limb bones is achieved through deep learning segmentation methods, including: Training a deep learning segmentation network for 3D lower limb bones; The deep learning segmentation network is used to classify pixels in the three-dimensional space into two categories according to whether they are in the lower limb bone region; if the pixel is in the non-lower limb bone region, the value is assigned to 0; if the pixel is in the lower limb bone region, the value is assigned to 1; The binary classification results are used as the three-dimensional lower limb bone voxel model.

4. A fully automatic measurement method for knee valgus and valgus according to claim 3, characterized in that: The deep learning segmentation network for training three-dimensional lower limb bones includes: The whole lower limb sequence consisting of multiple tomographic images was forward projected in the coronal plane to form a two-dimensional forward projection image; Use a two-dimensional U-Net to roughly segment the two-dimensional orthographic projection image and roughly locate each bone; The tomographic image is put into a three-dimensional U-Net with multiple layers of images as a group of images for fine bone segmentation, and the result of the rough positioning of each bone is referred to during the segmentation.

5. A fully automatic measurement method for knee valgus and valgus according to claim 4, characterized in that: The method further includes: Overlapping tomograms are provided between adjacent group images; After the segmentation of the adjacent group images is completed, the segmentation results of the overlapping tomograms are subjected to intersection fusion; if the bone voxel appears in the overlapping area of ​​the two groups at the same time, the bone voxel is retained, otherwise it is deleted.

6. A fully automatic measurement method for knee valgus and valgus according to claim 1, characterized in that: The reconstruction of lower limb bones is achieved by extracting isosurfaces or lack of model registration.

7. A fully automatic measurement method for knee valgus and valgus according to claim 6, characterized in that: The reconstruction of lower limb bones is achieved by extracting isosurfaces, including: Let the threshold of the desired isosurface be M; Construct multiple N×N×N cube voxels with each pixel in the three-dimensional lower limb bone voxel model obtained after segmentation as the center; Compare the pixel values ​​at each vertex of the cube voxel with a threshold M, and select the first voxel intersecting with the isosurface; The intersection points of the isosurface and the edges of the first volume element are obtained by interpolation, and the intersection points are connected into triangles to form an isosurface patch. The three-dimensional lower limb bone network model is formed by the collection of triangles in all the first volume elements.

8. The fully automatic measurement method of knee valgus and valgus according to claim 1, characterized in that: The step of extracting lower limb bone surface data points comprises: Performing connectivity detection on the three-dimensional lower limb bone network model to obtain a plurality of interconnected domains that are not connected to each other; Calculate the center coordinates of each of the connected domains; The lower limb bones are classified into left femur, right femur, left tibia, right tibia, left talus and right talus according to the central coordinates of the connected domain, and the surface data points of each lower limb bone are retained.

9. A fully automatic measurement method for knee valgus and valgus according to claim 8, characterized in that: The calculating the center coordinates of each of the connected regions includes: Each pixel in the three-dimensional space is classified into two categories according to whether it is inside the connected domain, and the average coordinates of all pixels inside the connected domain are counted to obtain the central coordinates of the connected domain.

10. The fully automatic measurement method of knee valgus and valgus according to claim 8, characterized in that: The method further includes: Set volume thresholds; Calculating the volume of each connected domain, where the volume of the connected domain is the sum of the number of pixels inside the connected domain; If the volume of the connected domain is greater than the volume threshold, the corresponding connected domain is identified as a valid connected domain; and if the volume of the connected domain is less than the volume threshold, the corresponding connected domain is identified as noise.

11. A fully automatic measurement method for knee valgus and valgus according to claim 1, characterized in that: The extraction process of the center point of the femoral head is specifically as follows: The data points of the proximal femur and the medial femur are selected as the femoral head data points; The femoral head data point with the highest height is selected as the femoral head apex and the femoral head data point closest to the inner side of the human body is selected as the medial point of the femoral head, and the initial femoral head center is determined by combining the Z-axis coordinate of the medial point of the femoral head and the X-axis coordinate and the Y-axis coordinate of the femoral head apex; Calculating first distances from the initial femoral head center to the femoral head apex and the medial point of the femoral head respectively, and taking the average of the two calculated first distances as a rough estimated radius of the femoral head; The second distance from the femoral head data point to the initial femoral head center is calculated, and spherical fitting is performed on the femoral head data points whose second distance is less than 1.2 times the rough estimated radius to obtain the accurately determined femoral head center point.

12. The fully automatic measurement method of knee valgus and valgus according to claim 1, characterized in that: The extraction process of the center point of the femoral head is specifically as follows: Select multiple sections at different positions or directions of the proximal femur, and perform center detection on the two-dimensional circular structure of the femoral head in the sections respectively; And, the center of a sphere in three-dimensional space is calculated using multiple circle centers to obtain the femoral head center.

13. A fully automatic method for measuring knee valgus and valgus according to claim 1, characterized in that: The extraction process of the knee joint center point is specifically as follows: Select the data points at the proximal end of the tibia as the knee joint data points; Selecting the knee joint data point closest to the inner side of the human body as the inner side of the knee joint point and the knee joint data point closest to the outer side of the human body as the outer side of the knee joint point; The coordinates of the medial point of the knee joint and the lateral point of the knee joint are averaged to obtain the center point of the knee joint.

14. A fully automatic method for measuring knee valgus and valgus according to claim 1, characterized in that: The extraction process of the knee joint center point is specifically as follows: Select the data points at the proximal end of the tibia as the knee joint data points; The coordinates of all the knee joint data points are averaged to obtain the center point of the knee joint.

15. The fully automatic measurement method of knee valgus and valgus according to claim 1, characterized in that: The extraction process of the center point of the ankle joint surface is specifically as follows: The data points at the proximal end of the talus were selected as the ankle joint data points; The ankle joint data point closest to the inside of the human body is selected as the inner ankle joint point, and the ankle joint data point closest to the outside of the human body is selected as the outer ankle joint point; The coordinates of the medial point of the ankle joint and the lateral point of the ankle joint are averaged to obtain the center point of the ankle joint.

16. A fully automatic method for measuring knee valgus and valgus according to claim 1, characterized in that: The center point of the ankle joint surface adopts the center point of the inner and outer malleolus.

17. The fully automatic measurement method of knee valgus and valgus according to claim 1, characterized in that: If the HKA angle is a two-dimensional angle, the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis are both composed of the coordinates of the X-axis and the Z-axis; And / or, if the HKA angle is a three-dimensional angle, the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis are both composed of coordinates of the X-axis, the Y-axis, and the Z-axis.

18. A fully automatic measurement system for knee valgus and valgus, characterized in that: The system is used to implement a fully automatic measurement method for knee valgus and valgus as described in any one of claims 1 to 17, comprising: A data acquisition module, used for acquiring three-dimensional lower limb bone data; A surface extraction module, used to segment and reconstruct the lower limb bones according to the three-dimensional lower limb bone data to obtain a three-dimensional lower limb bone network model, and extract lower limb bone surface data points; A center detection module, used to extract key points including the center point of the femoral head, the center point of the knee joint and the center point of the ankle joint surface from the lower limb bone surface data points, and determine the femoral mechanical axis and the tibial mechanical axis based on the key points; The angle calculation module is used to perform arc cosine solution on the dot product between the unit vector of the femoral mechanical axis and the unit vector of the tibial mechanical axis to calculate the HKA angle.

19. A computer terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements a fully automatic measurement method for knee valgus and valgus as described in any one of claims 1-17.

20. A computer readable medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement a fully automatic measurement method for knee valgus and valgus as described in any one of claims 1-17.

Images