Lower limb force line high-precision measurement system and method based on visual navigation

Abstract

The invention discloses a lower limb force line high-precision measurement system and method based on visual navigation, and particularly relates to the technical field of medical visual aided navigation, and the system is composed of a navigation processing computer, a measurement camera and a knee joint section characterization feature cursor tool. Firstly, a measuring camera shoots a cursor tool and transmits an image to a navigation processing computer, and a preprocessed image is obtained through operation such as image correction. Secondly, identifying, repairing and positioning feature cursors in the preprocessed image by adopting an intelligent learning algorithm to obtain coordinates of each feature cursor in a coordinate system of a measurement camera, and obtaining an ideal knee joint section equation and a knee joint section equation by measuring and estimating coordinates of a knee joint center and a femoral head center and the length of a mechanical axis for multiple times; according to the spatial position relation of the two sections, obtaining the inward and outward turning angle and the forward and backward inclination angle of the lower limb force line. The measuring system can be repeatedly used, and is high in measuring precision and high in popularization value.

Description

Technical Field

[0001] The present invention relates to the technical field of medical vision-assisted navigation, and in particular to a high-precision measurement system and method for lower limb force lines based on vision navigation. Background Art

[0002] The knee joint is the largest and most complex joint in the human body, consisting of the tibia-femoral joint and the patellar joint. It is also a vulnerable part. If not properly treated, the integrity of the structure and function of the articular cartilage may be damaged, and even lead to diseases such as knee arthritis, which has become a major problem in the quality of life of young, middle-aged and elderly people.

[0003] With the continuous progress of society and the improvement of material conditions, adults' requirements for quality of life are gradually increasing, and the demand for knee surgery will continue to increase. Therefore, in order to obtain good clinical results, the mastery of surgical skills plays a very critical role. How to accurately perform osteotomy in three dimensions is not only the first key step to the success of the operation, but also an important reason affecting its clinical effect. It is also a problem that orthopedic surgeons must face and solve.

[0004] The measurement accuracy of the force line of the lower limbs in total knee replacement surgery has a great impact on the patient's rehabilitation and quality of life in the later period. There are currently three solutions for the measurement of the force line of the lower limbs: the first method is to roughly estimate the size of the varus and valgus angles and the anteroposterior tilt angles of the force line of the lower limbs based on the experience of orthopedic surgeons over the years. This requires doctors to have rich surgical experience, and this estimation method has large deviations, and is also prone to poor treatment effects, causing symptoms of postoperative recurrence. The second method relies on preoperative X-ray examinations and intraoperative mechanical guide devices for intramedullary and extramedullary positioning osteotomies, which increases the risks of infection, increased bleeding, and embolism. The mechanical guide device cannot accurately measure the varus and valgus angles and the anteroposterior tilt angles of the force line of the lower limbs. The accuracy of the above angles is about 1°. The mechanical guide device is highly traumatic, which is not conducive to postoperative rehabilitation and has high surgical costs. The third method relies on a MEMS navigation device, which contains a gyroscope and an accelerometer. The MEMS navigation device is fixed on a fixture bracket. During the operation, the knee joint is rotated to generate multiple instantaneous accelerations and angular velocities in different directions, thereby obtaining the varus and valgus angles and the anteroposterior tilt angles. MEMS navigation devices are disposable products, resulting in high surgical costs and an angle measurement accuracy of approximately 0.5° to 1°. Therefore, it is of great significance to develop a low-cost, high-precision method for measuring lower limb force lines. Summary of the invention

[0005] To this end, the present invention provides a high-precision measurement system and method for lower limb force lines based on vision navigation to solve the problems raised in the background technology.

[0006] In order to achieve the above-mentioned purpose, the present invention provides the following technical solutions: a method for high-precision measurement of lower limb force lines based on visual navigation, based on a navigation processing computer, a measurement camera and a knee joint cross-section characterization feature cursor tool, the measurement method comprises the following steps:

[0007] (1) Calibrate the measurement camera and obtain camera calibration parameters;

[0008] (2) The measuring camera masks and captures the image and performs calculations to reduce invalid pixels and improve calculation accuracy;

[0009] (3) During the operation, the hip joint is fixed and the knee joint is moved in multiple directions, and the characteristic cursor tool of the knee joint cross section is used to take pictures using the measurement camera;

[0010] (4) The image captured in step (3) is transmitted to the navigation processing computer, grayscale processing is performed on the captured image, and a reasonable threshold is selected and binarized to obtain a preprocessed image;

[0011] (5) The cursor in the image preprocessed in step (4) has been separated from the background, and the cursor information is obtained after rough recognition;

[0012] (6) During the operation, the characteristic cursor is incomplete due to bloodstains and other obstructions. Therefore, a deep learning algorithm is used to identify the cursor, and transfer learning is used to repair the identified image and complete the image features to achieve precise positioning and extract the cursor position information;

[0013] (7) estimating the center coordinates of the knee joint by multiple measurements based on the cursor position information extracted in step (6);

[0014] (8) The coordinate information of the cursor in the coordinate system of the measuring camera is obtained by multiple measurements, and the motion trajectory of the characteristic cursor in different directions is obtained. A spherical equation with the center of the femoral head as the sphere center and the length of the mechanical axis as the radius is constructed, and the spherical surface is fitted using the least squares method;

[0015] (9) According to the spherical equation fitted in step (8), the center coordinates of the femoral head (x c ,y c ,z c ) and the mechanical axis length l, determine whether the solution result converges; if the convergence accuracy is not less than 0.1°, retake the image and repeat the above steps; if the accuracy requirement is met, calculate the ideal knee joint cross-section equation;

[0016] (10) Select the coordinates of a set of feature cursors with the smallest error and estimate the cross-sectional equation of the knee joint;

[0017] (11) Based on the spatial position relationship between the two planes of the knee joint cross section and the ideal knee joint cross section, the angle between the two planes, i.e., the varus and valgus angle, is calculated; based on the same principle, the anterior-posterior inclination angle can be calculated by rotating at different angles.

[0018] Preferably, in step (7), an iterative algorithm is used to obtain the cursor tool coordinate system, the displacement T and the posture rotation matrix R relative to the measurement camera coordinate system, and the knee joint center coordinates (x0, y0, z0) are estimated.

[0019] Preferably, the line connecting the center coordinates of the knee joint and the center coordinates of the femoral head is used as the normal line of the ideal knee joint cross section, and the ideal knee joint cross section equation Ax+By+Cz+D=0 is obtained, where x, y, z are the coordinates of any point on the ideal knee joint cross section, and A, B, C, D are the coefficients of the ideal knee joint cross section equation;

[0020] Select a set of feature cursors with the smallest error to estimate the knee joint cross-section equation A1x'+B1y'+C1z'+D1=0, where x', y', z' are any points on the knee joint cross-section, A1, B1, C1, D1 are the coefficients of the knee joint cross-section equation; the angle α between the projection vector n of the knee joint cross-section normal on the coronal plane and the mechanical axis is the varus angle or valgus angle, and the angle β between the projection vector m of the knee joint cross-section normal on the median plane and the mechanical axis is the anteversion angle or posterior tilt angle; the specific calculation is as follows:

[0021] The spherical equation after fitting is:

[0022] (xx c ) 2 +(yy c ) 2 +(zz c ) 2 -l 2 =0;

[0023] The linear equation of the mechanical axis is:

[0024]

[0025] Ideal knee joint cross-section equation:

[0026] (x c -x0)x+(y c -y0)y+(z c -z0)z+x0 2 -x0x c +y0 2 -y0y c +z0 2 -z0z c =0;

[0027] The equivalent is: Ax+By+Cz+D=0;

[0028] Through multiple measurements, the knee joint cross-section equation is calculated based on R and T: A1x'+B1y'+C1z'+D1=0;

[0029] Therefore, the varus or valgus angle α is:

[0030] α=arccos<n,[x c -x0,y c -y0,z c -z0,] T >;

[0031] The forward or backward inclination angle β is:

[0032] b=arccos<m,[x c -x0,y c -y0,z c -z0] T >;

[0033] Where <,> represents the angle between two vectors and T is the transpose.

[0034] The present invention also discloses a high-precision measurement system for lower limb force lines based on vision navigation, and the system is based on the above-mentioned high-precision measurement method for lower limb force lines based on vision navigation.

[0035] Beneficial Effects

[0036] 1. Image segmentation technology: In the field of computer vision navigation, image segmentation algorithms are used to pre-process feature cursor information, and then it is refined according to the region of interest. Deep learning is used to accurately locate the feature cursor, so that the feature cursor can be identified faster and more accurately, and it is not easy to lose frames.

[0037] 2. Depth of field iteration algorithm: The system adopts a high-precision posture solution method to improve the overall performance of the system. The force line measurement accuracy is less than 0.1°. Compared with the existing technology, it meets the requirements of high-precision lower limb force line measurement methods.

[0038] 3. The present invention takes into account the processing method when the feature cursor is blocked or lost. When the cursor is blocked, deep learning is used to recognize the image. Through the transfer learning method, the recognized image is repaired and the image features are completed. The trained model is used to extract and identify the feature cursor. Compared with the prior art, the advantage is that the fault tolerance of the system is increased and the recognition accuracy of the feature cursor is improved.

[0039] 4. The present invention is reusable, which greatly reduces patient expenses and complies with the national low-carbon environmental protection concept. It has low development cost, strong durability, strong robustness, high cost performance, low requirements on the working environment, and strong reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of a high-precision measurement system of lower limb force lines based on visual navigation of the present invention;

[0041] Figure 2 It is a schematic diagram of the definition of the force line parameters of the lower limbs of the present invention;

[0042] Figure 3 A geometric diagram for installing the coordinate systems and feature cursors of the present invention;

[0043] Figure 4 The present invention is a flow chart for calculating the inward and outward valgus angles and the forward and backward inclination angles of the lower limb force lines. DETAILED DESCRIPTION

[0044] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Please refer to the attached Figure 1 The high-precision measurement system of lower limb force lines based on visual navigation includes a navigation processing computer, a measuring camera, and a knee joint cross-section characterization feature cursor tool (referred to as the cursor tool). The cursor tool consists of multiple non-coplanar concentric circle cursors, which are fixed to the patient's knee joint through a clamp bracket, and the patient's hip joint is fixed. The knee joint is moved in multiple directions, and the coordinates of the center point of the knee joint and the knee joint cross-section equation are estimated through multiple measurements. The measuring camera is located directly above it, takes a picture of the cursor tool and transmits the image to the navigation processing computer. The navigation processing computer solves the cursor coordinates, surface fitting, internal and external varus angles, and anteroposterior inclination angles. The accuracy of determining the internal and external varus angles and anteroposterior inclination angles of the lower limb force lines can reach about 0.1°.

[0046] Please refer to the attached Figure 2 , defines the force line parameters of the lower limb. The femoral head center coordinates (x c ,y c ,z c), knee joint center coordinates (x0, y0, z0), the mechanical axis is the line between the femoral head center and the knee joint center. The anatomical axis is the central axis of the medullary cavity from the proximal femur to the distal femur. The ideal knee joint section equation is Ax+By+Cz+D=0, the knee joint section equation is A1x'+B1y'+C1z'+D1=0, the angle a between the projection vector n of the knee joint section normal on the coronal plane and the mechanical axis is the varus (external) angle, and the angle β between the projection vector m of the knee joint section normal on the median plane and the mechanical axis is the anterior (posterior) tilt angle. α1 is the valgus angle, and α2 is the varus angle. β1 is the anteversion angle, and β2 is the posterior tilt angle. The basis for distinguishing between the internal and external varus surfaces is to judge based on the normal of the internal and external varus surfaces. When the normal of the knee joint section points to the outside of the body, it is the internal varus surface, and vice versa, it is the external varus surface. Similarly, the anterior and posterior tilt surfaces are also the same.

[0047] Please refer to the attached Figure 3 , Figure 3 The image plane coordinate system O is defined i -X i Y i And the camera coordinate system O c -X c Y c Z c The image plane coordinate system is used to describe the physical coordinates of the projection point in the imaging plane. The origin of the image plane coordinate system is O i Located at the intersection of the camera axis and the imaging plane (usually at the center of the imaging plane), X i Axis and Y i The axes are parallel to the U and V axes of the pixel coordinate system. The origin of the camera coordinate system is O c Located at the center of the camera, Z c The axis is perpendicular to the imaging plane, X c Axis and Y c The axes are parallel to the X i Axis and Y i Axis, X c , Y c and Z c The axes form a right-handed coordinate system. The coordinates of the feature cursor in the camera coordinate system are defined as P c,n =[x c,n ,y c,n ,z c,n ] T (n=1,2,...). The coordinates of the feature cursor in the camera coordinate system and the center of the knee joint have the following geometric features: Cursor 1 p c，1 The line connecting the knee joint center and cursor 2 c，2 The angle with the knee joint is 30°, cursor 3 p c，3 The line connecting the knee joint is perpendicular to the cursor No. 4c，4 The line connecting the knee joint, cursor No. 4 c，4 The line connecting the knee joint and the cursor No. 5 c，5 The angle between the line connecting the knee joint and the knee joint is 90°. Similarly, the five feature cursors also have a certain geometric relationship with the knee joint installation section: No. 4 p c，4 and cursor No. 5 p c，5 Located in the plane of the knee joint section, cursor 2 p c，2 and cursor 3 p c，3 The line connecting the five feature cursors is a normal line of the knee joint cross section plane. Therefore, according to the positions of the five feature cursors, the coordinates of the center point of the knee joint and the plane equation of the knee joint cross section can be estimated in the camera coordinate system. Figure 3 The femoral head center coordinates (x c ,y c ,z c ) is the center of the sphere and the mechanical axis length l is the radius of the sphere. Based on the coordinates of the center of the knee joint and the spherical equation, the coordinates of the center of the femoral head are estimated through multiple measurements, so the linear equation of the mechanical axis can be obtained, and then the plane equation of the ideal knee joint section can be obtained.

[0048] Please refer to the attached Figure 4 , Figure 4 The flow chart of the high-precision measurement method of lower limb force alignment based on vision navigation is introduced.

[0049] (1) Calibrate the measurement camera to obtain the camera calibration parameters.

[0050] (2) The measuring camera masks the captured image and performs calculations to reduce invalid pixels and improve calculation accuracy.

[0051] (3) During the operation, the hip joint is fixed and the knee joint is moved in multiple directions, and the feature cursor tool is photographed using a measurement camera.

[0052] (4) The captured image is transmitted to the navigation processing computer, grayscale processed, and binarized after selecting a reasonable threshold to obtain a preprocessed image.

[0053] (5) The cursor in the preprocessed image has been separated from the background, and the cursor information is obtained after rough recognition.

[0054] (6) During the operation, the characteristic cursor is incomplete due to bloodstains and other obstructions. Therefore, a deep learning algorithm is used to identify the cursor, and transfer learning is used to repair the recognized image, complete the image features, achieve precise positioning, and extract the cursor position information.

[0055] (7) Based on the cursor position information, multiple measurements are performed to estimate the center coordinates of the knee joint.

[0056] (8) The coordinate information of the cursor in the coordinate system of the measuring camera is obtained by multiple measurements, and the motion trajectory of the characteristic cursor in different directions is obtained. A spherical equation is constructed with the center of the femoral head as the sphere center and the mechanical axis length as the radius, and the spherical fitting is performed using the least squares method.

[0057] (9) According to the fitted spherical equation, the center coordinates of the femoral head (x c ,y c ,z c ) and the mechanical axis length l, and determine whether the solution result converges. If it does not meet the convergence accuracy requirements, retake the image and repeat the above steps. If it meets the accuracy requirements, calculate the ideal knee joint cross-section equation.

[0058] (10) According to the line connecting the center coordinates of the knee joint and the center coordinates of the femoral head, the normal line of the ideal knee joint section is obtained, and the ideal knee joint section equation Ax+By+Cz+D=0 is obtained, where x, y, z are the coordinates of any point on the ideal knee joint section, and A, B, C, D are the coefficients of the ideal knee joint section equation. Select a set of feature cursors with the smallest error to estimate the knee joint section equation A1x'+B1y'+C1z'+D1=0, where x', y', z' are any points on the knee joint section, and A1, B1, C1, D1 are the coefficients of the knee joint section equation. The angle a between the projection vector n of the knee joint section normal on the coronal plane and the mechanical axis is the inward (outward) varus angle, and the angle β between the projection vector m of the knee joint section normal on the median plane and the mechanical axis is the anterior (posterior) inclination angle. The specific calculation is as follows:

[0059] The spherical equation after fitting is:

[0060] (xx c ) 2 +(yy c ) 2 +(zz c ) 2 -l 2 =0;

[0061] The linear equation of the mechanical axis is:

[0062]

[0063] Ideal knee joint cross-section (coronal plane) equation:

[0064] (x c -x0)x+(y c -y0)y+(z c -z0)z+x0 2 -x0x c +y0 2 -y0y c +z02 -z0z c =0;

[0065] The equivalent is: Ax+By+Cz+D=0;

[0066] Through multiple measurements, the knee joint cross-sectional equation was calculated: A1x'+B1y'+C1z'+D1=0;

[0067] Therefore, the inward (outward) varus angle α is:

[0068] a=arccos<n,[x c -x0,y c -y0,z c -z0] T >;

[0069] The front (rear) inclination angle β is:

[0070] b=arccos<m,[x c -x0,y c -y0,z c -z0] T >;

[0071] Where <,> represents the angle between two vectors and T is the transpose.

[0072] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A high-precision measurement method for lower limb force lines based on visual navigation, based on a navigation processing computer, a measurement camera, and a knee joint cross-section characterization feature cursor tool, characterized by: The measurement method includes the following steps: (1) Calibrate the measurement camera and obtain camera calibration parameters; (2) The measuring camera masks the captured image and performs calculations to reduce invalid pixels; (3) During the operation, the hip joint is fixed and the knee joint is moved in multiple directions, and the measurement camera is used to take pictures of the knee joint cross-section characterization feature cursor tool; (4) The image captured in step (3) is transmitted to the navigation processing computer, grayscale processing is performed on the captured image, and a reasonable threshold is selected and binarized to obtain a preprocessed image; (5) The cursor in the image preprocessed in step (4) has been separated from the background, and the cursor information is obtained after rough recognition; (6) During the operation, a deep learning algorithm is used to identify the cursor, and transfer learning is used to repair the identified image and complete the image features to achieve precise positioning; (7) estimating the center coordinates of the knee joint by multiple measurements based on the cursor position information extracted in step (6); (8) The coordinate information of the cursor in the coordinate system of the measuring camera is obtained by multiple measurements, and the motion trajectory of the characteristic cursor in different directions is obtained. A spherical equation with the center of the femoral head as the sphere center and the length of the mechanical axis as the radius is constructed, and the spherical surface is fitted using the least squares method; (9) According to the spherical equation fitted in step (8), the center coordinates of the femoral head (x c ,y c ,z c ) and the mechanical axis length l, determine whether the solution result converges; if the convergence accuracy is not less than 0.1°, retake the image and repeat the above steps; if the accuracy requirement is met, calculate the ideal knee joint cross-section equation; (10) Select the coordinates of a set of feature cursors with the smallest error and estimate the cross-sectional equation of the knee joint; (11) Based on the spatial position relationship between the two planes of the knee joint cross section and the ideal knee joint cross section, the angle between the two planes, i.e., the varus and valgus angle, is calculated; based on the same principle, the anterior-posterior inclination angle can be calculated by rotating at different angles.

2. According to claim 1, a method for high-precision measurement of lower limb force lines based on visual navigation is characterized in that: In the step (7), an iterative algorithm is used to obtain the cursor tool coordinate system, the displacement T and the posture rotation matrix R relative to the measurement camera coordinate system, and the knee joint center coordinates (x0, y0, z0) are estimated.

3. The method for high-precision measurement of lower limb force lines based on visual navigation according to claim 2 is characterized in that: The line connecting the center coordinates of the knee joint and the center coordinates of the femoral head is used as the normal line of the ideal knee joint cross section, and the ideal knee joint cross section equation is obtained as Ax+By+Cz+D=0, where x, y, z are the coordinates of any point on the ideal knee joint cross section, and A, B, C, D are the coefficients of the ideal knee joint cross section equation; Select a set of feature cursors with the smallest error to estimate the knee joint cross-section equation A1x'+B1y'+C1z'+D1=0, where x', y', z' are any points on the knee joint cross-section, A1, B1, C1, D1 are the coefficients of the knee joint cross-section equation; the angle α between the projection vector n of the knee joint cross-section normal on the coronal plane and the mechanical axis is the varus angle or valgus angle, and the angle β between the projection vector m of the knee joint cross-section normal on the median plane and the mechanical axis is the anteversion angle or posterior tilt angle; the specific calculation is as follows: The spherical equation after fitting is: (x-x c ) 2 +(y-y c ) 2 +(z-z c ) 2 -l 2 =0; The linear equation of the mechanical axis is: Ideal knee joint cross-section equation: (x c −x0)x+(y c -y0)y+(z c -z0)z+x0 2 -x0x c +y0 2 -y0y c +z0 2 -z0z c =0: The equivalent is: Ax+By+Cz+D=0; Through multiple measurements, the knee joint cross-section equation is calculated based on R and T: A1x'+B1y'+C1z'+D1=0; Therefore, the varus or valgus angle α is: α=arccos<n,[x c −x0,y c −y0,z c −z0,] T  The forward or backward inclination angle β is: b=arcs<m,[x c −x0,y c −y0,z c −z0] T  Where <,> represents the angle between two vectors and T is the transpose.

4. A high-precision measurement system for lower limb force lines based on visual navigation, characterized in that: The system is based on a high-precision measurement method of lower limb force lines based on visual navigation according to any one of claims 1-3.

Images