A high-precision measurement system and method for lower limb force lines based on vision navigation

Abstract

This invention discloses a high-precision measurement system and method for lower limb force lines based on visual navigation, specifically relating to the field of medical visual-assisted navigation technology. The system consists of a navigation processing computer, a measuring camera, and a feature cursor tool representing the knee joint cross-section. First, the measuring camera captures an image of the cursor tool and transmits it to the navigation processing computer. After image correction and other operations, a preprocessed image is obtained. Second, an intelligent learning algorithm is used to identify, repair, and locate the feature cursors in the preprocessed image, obtaining the coordinates of each feature cursor in the measuring camera's coordinate system. Through multiple measurements, the coordinates of the knee joint center and femoral head center, as well as the mechanical axis length, are estimated, yielding the ideal knee joint cross-section equation and the knee joint cross-section equation. Based on the spatial relationship between the two cross-sections, the varus / valgus angle and anteroposterior / posterior tilt angle of the lower limb force line are obtained. This measurement system is reusable, has high measurement accuracy, and has significant potential for widespread application.

Description

Technical Field

[0001] This invention relates to the field of medical vision-assisted navigation technology, specifically to a high-precision measurement system and method for lower limb force lines based on vision navigation. Background Technology

[0002] The knee joint is the largest and most complex joint in the human body, composed of the tibiofemoral joint and the patellar joint, and it is also a vulnerable area. Improper care can damage the integrity of the articular cartilage structure and function, even leading to diseases such as knee arthritis, which has become a major concern for the quality of life of young, middle-aged, and elderly people.

[0003] With the continuous progress of society and the enrichment of material conditions, adults are gradually increasing their demands for quality of life, and the demand for surgery for knee joint patients will continue to increase. Therefore, in order to achieve good clinical results, mastering surgical techniques plays a very important role. How to accurately perform osteotomy in three dimensions is not only the first key step to surgical success, but also an important factor affecting its clinical effect, and a problem that orthopedic surgeons must face and solve.

[0004] The accuracy of lower limb alignment measurement during total knee replacement surgery has a significant impact on postoperative patient recovery and quality of life. Currently, there are three approaches to measuring lower limb alignment: The first approach involves roughly estimating the varus / valgus and anteversion / anteroversion angles based on the orthopedic surgeon's years of experience. This requires extensive surgical experience and is prone to significant errors, leading to poor treatment outcomes and postoperative recurrence. The second approach relies on preoperative X-ray examination and intraoperative mechanical guidance devices for intramedullary and extramedullary osteotomy positioning. This increases the risks of infection, bleeding, and embolism. Furthermore, mechanical guidance devices cannot accurately measure the varus / valgus and anteversion / anteroversion angles, with an accuracy of approximately 1°. Mechanical guidance devices are also invasive, detrimental to postoperative recovery, and expensive. The third approach utilizes a MEMS navigation device, containing a gyroscope and accelerometer. The MEMS navigation device is fixed to a clamp support. During surgery, the knee joint rotates, generating instantaneous accelerations and angular velocities in multiple directions, thereby obtaining the varus / valgus and anteversion / anteroversion angles. MEMS navigation devices are disposable products, leading to high surgical costs, and their angular measurement accuracy is only about 0.5° to 1°. Therefore, developing a low-cost, high-precision method for measuring lower limb alignment is of great significance. Summary of the Invention

[0005] Therefore, the present invention provides a high-precision measurement system and method for lower limb force lines based on visual navigation to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a high-precision measurement method for lower limb force lines based on visual navigation, comprising a navigation processing computer, a measuring camera, and a cursor tool for characterizing knee joint cross-sections, the measurement method including the following steps:

[0007] (1) Calibrate the measuring camera and obtain the camera calibration parameters;

[0008] (2) The camera performs masking to capture images and performs calculations to reduce invalid pixels and improve calculation accuracy;

[0009] (3) During the operation, fix the hip joint and move the knee joint in multiple directions, and take pictures of the knee joint cross-section characterization cursor tool using a measuring camera;

[0010] (4) The image captured in step (3) is transmitted to the navigation processing computer, grayscale processing is performed on the captured image, and binarization is performed after selecting a reasonable threshold to obtain the preprocessed image.

[0011] (5) The cursor in the preprocessed image in step (4) has been separated from the background. After coarse recognition, the information of the cursor is obtained.

[0012] (6) During the operation, the feature cursor is incomplete due to bloodstains and other obstructions. Therefore, a deep learning algorithm is used to identify the cursor, and transfer learning is used to repair the identified image and complete the image features in order to achieve precise positioning and extract the position information of the cursor.

[0013] (7) Based on the cursor position information extracted in step (6), the coordinates of the knee joint center are estimated by multiple measurements;

[0014] (8) The coordinate information of the cursor in the coordinate system of the measuring camera is obtained by multiple measurements. The motion trajectory of the feature cursor in different directions is obtained. The spherical equation with the femoral head center as the center and the mechanical axis length as the radius is constructed. The least squares method is used for spherical fitting.

[0015] (9) Obtain the coordinates (x, y) of the femoral head center based on the spherical equation fitted in step (8). c ,y c ,z c The solution is determined by taking the mechanical axis length l and checking if the solution converges. If the convergence accuracy is less than 0.1°, the image is retaken and the above steps are repeated. If the accuracy requirement is met, the equation of the ideal knee joint section is calculated.

[0016] (10) Select the coordinates of the feature cursors with the smallest error and estimate the equation of the knee joint section;

[0017] (11) Based on the spatial relationship between the two planes of the knee joint section and the ideal knee joint section, the included angle between the two planes, i.e. the inward and outward turning angles, can be calculated by rotating them at different angles according to the same principle.

[0018] Preferably, in step (7), an iterative algorithm is used to obtain the displacement T and attitude rotation matrix R of the cursor tool coordinate system relative to the measurement camera coordinate system, and to estimate the center coordinates (x0, y0, z0) of the knee joint.

[0019] Preferably, the line connecting the center coordinates of the knee joint and the center coordinates of the femoral head is used as the normal to the ideal knee joint section, resulting in the ideal knee joint section equation Ax+By+Cz+D=0, where x, y, z are the coordinates of any point on the ideal knee joint section, and A, B, C, D are the coefficients of the ideal knee joint section equation.

[0020] The knee joint section equation A1x'+B1y'+C1z'+D1=0 is estimated by selecting the feature cursor with the smallest error, where x', y', and z' are any points on the knee joint section, and A1, B1, C1, and D1 are the coefficients of the knee joint section equation. The angle α between the projection vector n of the knee joint section normal onto the coronal plane and the mechanical axis is the varus or valgus angle, and the angle β between the projection m of the knee joint section normal onto the midline plane and the mechanical axis is the anteversion or kyphosis angle. The specific calculation is as follows:

[0021] The fitted equation of the sphere is:

[0022] (xx c ) 2 +(yy c ) 2 +(zz c ) 2 -l 2 =0;

[0023] The equation of the straight line of the mechanical axis is:

[0024]

[0025] Ideal knee joint cross-section equation:

[0026] (x c -x0)x+(y c -y0)y+(z c -z0)z+x0 2 -x0x c +y0 2 -y0y c +z0 2 -z0z c =0;

[0027] The equivalent is: Ax + By + Cz + D = 0;

[0028] Through multiple measurements, the equation of the knee joint section was calculated based on R and T: A1x'+B1y'+C1z'+D1=0;

[0029] Therefore, the inverted or everted angle α is:

[0030] α = arccos < n, [x c -x0,y c -y0,z c -z0,] T >;

[0031] The pitch angle or loin angle β is:

[0032] b = arccos < m, [x] c -x0,y c -y0,z c -z0] T >;

[0033] Where <,> represents the angle between two vectors, and T is the transpose.

[0034] The present invention also discloses a high-precision measurement system for lower limb force lines based on visual navigation, which is based on the above-mentioned high-precision measurement method for lower limb force lines based on visual navigation.

[0035] Beneficial effects

[0036] 1. Image segmentation technology: In the field of computer vision navigation, image segmentation algorithms are used to preprocess feature cursor information, and then the information is further subdivided according to the region of interest. Deep learning is used for precise localization of feature cursors, thereby recognizing feature cursors with faster and higher accuracy and less frame loss.

[0037] 2. Depth-of-field iterative algorithm: A high-precision pose calculation method is adopted, which improves the overall performance of the system. The force line measurement accuracy is less than 0.1°, which meets the requirements of high-precision lower limb force line measurement methods compared with existing technologies.

[0038] 3. This invention considers the handling methods when the feature cursor is occluded or lost. When the cursor is occluded, deep learning is used for image recognition. Through transfer learning, the recognized image is repaired and the image features are supplemented. The trained model is used to extract and identify the feature cursor. Compared with the prior art, the advantages are that it increases the fault tolerance of the system and improves the recognition accuracy of the feature cursor.

[0039] 4. This invention is reusable, greatly reduces patient expenses, conforms to the national low-carbon and environmental protection concept, has low development cost, strong durability, strong robustness, high cost performance, low requirements for working environment, and strong reliability. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the high-precision measurement system for lower limb force lines based on vision navigation according to the present invention;

[0041] Figure 2 This is a schematic diagram illustrating the definition of lower limb force line parameters in this invention;

[0042] Figure 3 This is a geometrical diagram illustrating the relationships between the various coordinate systems and the installation of the feature cursor in this invention;

[0043] Figure 4 This is a flowchart illustrating the calculation process for the inward and outward tilt angles and forward and backward tilt angles of the lower limb force line according to the present invention. Detailed Implementation

[0044] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Please refer to the appendix. Figure 1 The high-precision measurement system for lower limb force lines based on vision navigation comprises a navigation processing computer, a measuring camera, and a cursor tool for characterizing the knee joint cross-section (hereinafter referred to as the cursor tool). The cursor tool consists of multiple non-coplanar concentric circular cursors, fixed to the patient's knee joint via a clamp bracket, with the patient's hip joint also secured. The knee joint is moved in multiple directions, and through multiple measurements, the coordinates of the knee joint center point and the knee joint cross-section equation are estimated. The measuring camera is positioned directly above the cursor tool, capturing images and transmitting them to the navigation processing computer. The navigation processing computer calculates the cursor coordinates, surface fitting, varus / valgus angles, and anteversion / antiversion angles. The accuracy of determining the varus / valgus angles and anteversion / antiversion angles of the lower limb force lines can reach approximately 0.1°.

[0046] Please refer to the appendix. Figure 2 The lower limb force line parameters are defined. Among them, the coordinates of the femoral head center (x...) c ,y c ,z cThe coordinates of the knee joint center are (x0, y0, z0). The mechanical axis is the line connecting the center of the femoral head and the center of the knee joint. The anatomical axis is the central axis of the medullary canal from the proximal to the distal femur. The ideal knee joint cross-section equation is Ax + By + Cz + D = 0, and the knee joint cross-section equation is A1x' + B1y' + C1z' + D1 = 0. The angle α between the projection vector n of the knee joint cross-section normal on the coronal plane and the mechanical axis is the varus (valgus) angle. The angle β between the projection m of the knee joint cross-section normal on the midline plane and the mechanical axis is the anteversion (posterior) angle. α1 is the valgus angle, α2 is the varus angle. β1 is the anteversion angle, β2 is the posterior angle. The distinction between varus and valgus is based on the normals of the varus and valgus surfaces. When the normal of the knee joint cross-section points outward, it is varus; conversely, when it points inward, it is valgus. The same applies to anteversion and posterior anteversion.

[0047] Please refer to the appendix. Figure 3 , Figure 3 The image plane coordinate system O is defined. i -X i Y i and camera coordinate system O c -X c Y c Z c The image plane coordinate system is used to describe the physical coordinates of the projection point in the imaging plane. The origin O of the image plane coordinate system is... i Located at the intersection of the camera's principal axis and the imaging plane (generally at the center of the imaging plane), X i axis and Y i The axes are parallel to the U-axis and V-axis of the pixel coordinate system, respectively. The origin O of the camera coordinate system is... c Located at the center of the camera, Z c The axis is perpendicular to the imaging plane, X c axis and Y c The axes are parallel to the X and Y axes in the image plane coordinate system. i axis and Y i Axis, X c Y c and Z c The axes form a right-handed coordinate system. The coordinates of the feature cursor in the camera coordinate system are defined as P. c,n =[x c,n ,y c,n ,z c,n ] T (n = 1, 2, ...). The coordinates of the feature cursor in the camera coordinate system and the center of the knee joint have the following geometric characteristics: Cursor 1 p c，1 The line connecting the center of the knee joint and cursor p (number 2) c，2 The angle between the line connecting the knee joint and the point is 30°, and cursor number 3 is p. c，3 The line connecting the knee joint is perpendicular to cursor p (number 4).c，4 The line connecting to the knee joint, cursor number 4 p c，4 The line connecting the knee joint and cursor p (number 5) c，5 The angle between the line connecting the knee joint and the point of contact is 90°. Similarly, the five feature cursors also have a certain geometric relationship with the knee joint mounting section: 4. c，4 And cursor number 5 p c，5 Located in the plane of the knee joint section, and cursor number 2 p c，2 And cursor number 3 p c，3 The line connecting the five features is a normal to the plane of the knee joint section. Therefore, based on the positions of the five feature cursors, the coordinates of the knee joint center point and the plane equation of the knee joint section can be estimated in the camera coordinate system. Furthermore... Figure 3 It shows the coordinates of the femoral head center (x) c ,y c ,z c Imagine a sphere with center (l) and radius (l) of the mechanical axis. Based on the coordinates of the knee joint center and the equation of the sphere, the coordinates of the femoral head center are estimated through multiple measurements. Therefore, the linear equation of the mechanical axis can be obtained, leading to the planar equation of the ideal knee joint cross-section.

[0048] Please refer to the appendix. Figure 4 , Figure 4 A flowchart of a high-precision measurement method for lower limb force lines based on vision navigation is presented.

[0049] (1) Calibrate the measuring camera and obtain the camera calibration parameters.

[0050] (2) The measurement camera performs masking to capture images and performs calculations to reduce invalid pixels and improve calculation accuracy.

[0051] (3) During the operation, fix the hip joint and move the knee joint in multiple directions, and take pictures of the feature cursor tool using a measuring camera.

[0052] (4) The captured image is transmitted to the navigation processing computer, grayscale processing is performed on the captured image, and binarization is performed after selecting a reasonable threshold to obtain the preprocessed image.

[0053] (5) The cursor in the preprocessed image has been separated from the background, and the cursor information is obtained after coarse recognition.

[0054] (6) During the operation, the feature cursor is incomplete due to bloodstains and other obstructions. Therefore, a deep learning algorithm is used to identify the cursor, and transfer learning is used to repair the identified image, complete the image features, achieve precise positioning, and extract the position information of the cursor.

[0055] (7) Based on the position information of the cursor, the coordinates of the center of the knee joint are estimated by multiple measurements.

[0056] (8) The coordinate information of the cursor in the coordinate system of the measuring camera is obtained by multiple measurements. The motion trajectory of the feature cursor in different directions is obtained. The spherical equation with the center of the femoral head as the center and the length of the mechanical axis as the radius is constructed. The least squares method is used to fit the spherical surface.

[0057] (9) Obtain the coordinates (x, y) of the femoral head center based on the fitted spherical equation. c ,y c ,z c The solution is determined by taking the image and the mechanical axis length *l*, and then checking if the solution converges. If the convergence accuracy requirement is not met, the image is retaken, and the above steps are repeated. If the accuracy requirement is met, the equation of the ideal knee joint section is calculated.

[0058] (10) The line connecting the center coordinates of the knee joint and the center coordinates of the femoral head can be used as the normal to the ideal knee joint section, resulting in the ideal knee joint section equation Ax + By + Cz + D = 0, where x, y, z are the coordinates of any point on the ideal knee joint section, and A, B, C, D are the coefficients of the ideal knee joint section equation. The knee joint section equation A1x' + B1y' + C1z' + D1 = 0 is estimated by selecting the set of feature cursors with the smallest error, where x', y', z' are any point on the knee joint section, and A1, B1, C1, D1 are the coefficients of the knee joint section equation. The angle α between the projection vector n of the knee joint section normal onto the coronal plane and the mechanical axis is the medial (lateral) roll angle, and the angle β between the projection m of the knee joint section normal onto the midline plane and the mechanical axis is the anteversion (posterior) roll angle. The specific calculations are as follows:

[0059] The fitted equation of the sphere is:

[0060] (xx c ) 2 +(yy c ) 2 +(zz c ) 2 -l 2 =0;

[0061] The equation of the straight line of the mechanical axis is:

[0062]

[0063] Ideal knee joint cross-section (coronal plane) equation:

[0064] (x c -x0)x+(y c -y0)y+(z c -z0)z+x0 2 -x0x c +y0 2 -y0y c +z02 -z0z c =0;

[0065] The equivalent is: Ax + By + Cz + D = 0;

[0066] Through multiple measurements, the equation for the knee joint section was calculated as: A1x'+B1y'+C1z'+D1=0;

[0067] Therefore, the inward (outward) fold angle α is:

[0068] a = arccos < n, [x] c -x0,y c -y0,z c -z0] T >;

[0069] The fore-and-aft angle β is:

[0070] b = arccos < m, [x] c -x0,y c -y0,z c -z0] T >;

[0071] Where <,> represents the angle between two vectors, and T is the transpose.

[0072] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A high-precision measurement method for lower limb force lines based on visual navigation, comprising a navigation processing computer, a measuring camera, and a cursor tool representing the cross-sectional features of the knee joint, characterized by: The measurement method includes the following steps: (1) Calibrate the measuring camera and obtain the camera calibration parameters; (2) The camera performs masking to capture images and performs calculations to reduce invalid pixels; (3) During the operation, fix the hip joint and move the knee joint in multiple directions, and take pictures of the knee joint cross-section characterization cursor tool using a measuring camera; (4) The image captured in step (3) is transmitted to the navigation processing computer, grayscale processing is performed on the captured image, and binarization is performed after selecting a reasonable threshold to obtain the preprocessed image. (5) The cursor in the preprocessed image in step (4) has been separated from the background. After coarse recognition, the information of the cursor is obtained. (6) During the operation, a deep learning algorithm is used to identify the cursor, and transfer learning is used to repair the identified image and complete the image features in order to achieve precise localization; (7) Based on the cursor position information extracted in step (6), the coordinates of the knee joint center are estimated by multiple measurements; (8) The coordinate information of the cursor in the coordinate system of the measuring camera is obtained by multiple measurements. The motion trajectory of the feature cursor in different directions is obtained. The spherical equation with the femoral head center as the center and the mechanical axis length as the radius is constructed. The least squares method is used for spherical fitting. (9) Obtain the coordinates (x, y) of the femoral head center based on the spherical equation fitted in step (8). c ,y c ,z c The solution is determined by taking the mechanical axis length l and checking if the solution converges. If the convergence accuracy is less than 0.1°, the image is retaken and the above steps are repeated. If the accuracy requirement is met, the equation of the ideal knee joint section is calculated. (10) Select the coordinates of the feature cursors with the smallest error and estimate the equation of the knee joint section; specifically: The line connecting the center coordinates of the knee joint and the center coordinates of the femoral head is used as the normal to the ideal knee joint section, resulting in the ideal knee joint section equation Ax + By + Cz + D = 0, where x, y, z are the coordinates of any point on the ideal knee joint section, and A, B, C, D are the coefficients of the ideal knee joint section equation. The knee joint section equation A1x'+B1y'+C1z'+D1=0 is estimated by selecting the feature cursor with the smallest error, where x', y', z' are any points on the knee joint section, and A1, B1, C1, D1 are the coefficients of the knee joint section equation; the angle α between the projection vector n of the knee joint section normal on the coronal plane and the mechanical axis is the varus or valgus angle, and the angle β between the projection m of the knee joint section normal on the midline plane and the mechanical axis is the anteversion or kyphosis angle. (11) Based on the spatial relationship between the two planes of the knee joint section and the ideal knee joint section, the included angle between the two planes, i.e. the inward and outward turning angles, can be calculated by rotating them at different angles according to the same principle.

2. The high-precision measurement method for lower limb force lines based on vision navigation according to claim 1, characterized in that: In step (7), an iterative algorithm is used to obtain the cursor tool coordinate system, the displacement T and attitude rotation matrix R relative to the measurement camera coordinate system, and to estimate the knee joint center coordinates (x0, y0, z0).

3. The high-precision measurement method for lower limb force lines based on vision navigation according to claim 2, characterized in that: The fitted equation of the sphere is: (x-x c ) 2 +(y-y c ) 2 +(z-z c ) 2 -l 2 =0; The equation of the straight line of the mechanical axis is: Ideal knee joint cross-section equation: (x c −x0)x+(y c -y0)y+(z c -z0)z+x0 2 -x0x c +y0 2 -y0y c +z0 2 -z0z c =0: The equivalent is: Ax + By + Cz + D = 0; Through multiple measurements, the equation of the knee joint section was calculated based on R and T: A1x'+B1y'+C1z'+D1=0; Therefore, the inverted or everted angle α is: α=arccos<n,[x c −x0,y c −y0,z c −z0,] T  The pitch angle or loin angle β is: b=arcs<m,[x c −x0,y c -y0,z c −z0] T  Where <,> represents the angle between two vectors, and T is the transpose.

4. A high-precision measurement system for lower limb force lines based on vision navigation, characterized in that: The system is based on a high-precision measurement method for lower limb force lines based on vision navigation as described in any one of claims 1-3.

Images