Axis estimation method and axis estimation device

Abstract

To provide an axis estimation method and axis estimation device accurately estimating an axis of a tibia.SOLUTION: An axis estimation method includes: a shin point group detection step of detecting point group data relating to the shin based on template data representing a knee joint of a leg part from the point group data relating to the leg part including the knee and shin; a protrusion determination step of determining, for each of a plurality of transverse planes perpendicular to the longitudinal direction of the shin out of the point group data relating to the shin, a protrusion of the shin in a traverse plane from a plurality of points corresponding to the traverse planes; an estimation step of estimating an axis of the tibia of the shin based on the plurality of protrusions.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an axis estimation method and an axis estimation device.

Background Art

[0002] In the field of motion and movement analysis, it is common to use motion capture for measuring knee movement (for example, Non-Patent Document 1). Also, for measuring the instability of the knee joint, measurement is performed by applying a load to the tibia and patella (for example, Non-Patent Document 2).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, motion capture makes it difficult to measure movements on the order of a few millimeters or less than 1 degree. Furthermore, measuring the instability of the tibia and patella by applying load, as shown in Non-Patent Literature 2, can only measure the displacement of one degree of freedom, and cannot measure all six degrees of freedom. The object of the present invention is to provide an axis estimation method and an axis estimation apparatus that solve the above-mentioned problems. [Means for solving the problem]

[0005] One aspect of the present invention is an axis estimation method comprising: a shin point cloud detection step of detecting point cloud data relating to the shin from point cloud data relating to the leg including the knee and shin, based on template data indicating the knee joint of the leg; a convexity determination step of determining a convex portion of the shin in each of a plurality of cross-sectional planes perpendicular to the longitudinal direction of the shin from a plurality of points corresponding to the cross-sectional plane among the point cloud data relating to the shin; and an estimation step of estimating the axis of the tibia of the shin based on the plurality of convex portions.

[0006] One aspect of the present invention is an axis estimation device comprising: a shin point cloud detection unit that detects point cloud data relating to the shin from point cloud data relating to the leg including the knee and shin, based on template data indicating the knee joint of the leg; a convexity determination unit that determines a convex portion of the shin in each of a plurality of cross-sectional planes perpendicular to the longitudinal direction of the shin from a plurality of points corresponding to the cross-sectional plane; and an estimation unit that estimates the axis of the tibia of the shin based on the plurality of convex portions. [Effects of the Invention]

[0007] According to the present invention, the axis of the tibia can be estimated more accurately. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows the axis estimation device and depth camera according to the first embodiment. [Figure 2] This is a diagram showing the configuration of the axis estimation device according to the first embodiment. [Figure 3] This figure shows the detected point cloud data. [Figure 4] This figure shows the point cloud data corresponding to each part. [Figure 5] This diagram shows a method for determining protrusions using circle detection. [Figure 6] This diagram shows the configuration of a knee joint template creation device. [Figure 7] This is an example of point cloud data related to the extracted knee joint. [Figure 8] This is a flowchart showing the operation of the axis estimation device. [Figure 9] This is a flowchart showing the operation of the knee joint template creation device. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described in detail below with reference to the drawings. <First Embodiment> Figure 1 shows the axis estimation device 1 and depth camera 10 according to the first embodiment. The axis estimation device 1 is connected to the depth camera 10. The depth camera 10 can dynamically acquire depth information of the object being photographed. The axis estimation device 1 acquires information from the depth camera 10 that captures a person P and estimates the axis of the person's tibia.

[0010] Figure 2 shows the configuration of the axis estimation device 1 according to the first embodiment. The axis estimation device 1 comprises a depth image acquisition unit 11, a point cloud data conversion unit 12, a knee joint detection unit 13, a partial point cloud detection unit 14, a convex part determination unit 15, an estimation unit 16, a knee joint template storage unit 17, and an angle measurement unit 18.

[0011] The depth image acquisition unit 11 acquires a depth image from the depth camera 10. Each pixel in the depth image is assigned a pixel value. The pixel value in the depth image represents the depth of the pixel to which that pixel value is assigned, that is, the distance between the optical element of the depth camera 10 and the object. In other words, the depth image can be said to represent the position of the object in a three-dimensional polar coordinate representation that shows the elevation angle (vertical axis), azimuth angle (horizontal axis), and distance (pixel value) from the origin. When the depth camera 10 takes a picture while the human legs are within the shooting range of the depth camera 10, the depth image acquired by the depth image acquisition unit 11 will include the human legs.

[0012] The point cloud data conversion unit 12 converts the depth image into point cloud data. Each point that makes up the point cloud data indicates its position in a three-dimensional Cartesian coordinate system. The point cloud data conversion unit 12 converts the depth image into point cloud data by, for example, projecting a three-dimensional polar coordinate representation that indicates the position of a pixel according to the magnitude of the pixel value of the depth image onto a three-dimensional Cartesian coordinate system.

[0013] The knee joint detection unit 13 detects knee joint point cloud data consisting of points related to the knee joint included in the point cloud data, based on the knee joint template stored in the knee joint template storage unit 17. Figure 3 shows the detected knee joint point cloud data. Of the point cloud data shown in Figure 3, the point cloud data shown in gray is point cloud data related to the leg, and the point cloud data enclosed by the dotted line is knee joint point cloud data related to the detected knee joint. The knee joint template may be a template obtained by photographing the knee joint of the same person in advance and may be point cloud data showing the characteristics of that person's knee joint, or it may be a template obtained by photographing the knee joint of any person without limiting it to a specific person and may be point cloud data showing the characteristics of the knee joint. The method for generating the knee joint template will be described later. The knee joint detection unit 13 detects knee joint point cloud data corresponding to the knee joint from the point cloud data by applying, for example, the RANSAC (Random Sample Consensus) method to the knee joint template and the point cloud data converted by the point cloud data conversion unit 12. Thereby, the position and orientation of the patella are determined. Further, the knee joint detection unit 13 can minimize the error caused by the RANSAC method by applying ICP (Iterative Closest Point) to the knee joint template and the point cloud data.

[0014] The partial point cloud detection unit 14 detects tibial partial point cloud data related to the tibia based on the knee joint point cloud data detected by the knee joint detection unit 13. More specifically, the partial point cloud detection unit 14 detects the tibial partial point cloud data by the following operations. First, the partial point cloud detection unit 14 performs principal component analysis on the knee joint point cloud data and determines that the direction in which the variance of the point cloud data is the smallest is the front direction of the patella. Then, among the point cloud data obtained by removing the knee joint point cloud data and the point cloud data in its boundary region from the point cloud data converted by the point cloud data conversion unit 12, the point cloud data located below the center of the knee joint is used as the tibial partial point cloud data. Also, the partial point cloud detection unit 14 uses the point cloud data located above the center of the knee joint as the femoral partial point cloud data. FIG. 4 is a diagram showing the point cloud data corresponding to each part. In FIG. 4, knee joint point cloud data 61, tibial partial point cloud data 62, and femoral partial point cloud data 63 are shown. The partial point group detection unit 14 can perform principal component analysis on the leg point group data and the shin point group data, and estimate the direction in which the variance is the largest as the direction in which the axis of each part extends. Here, the axes determined for the leg and the shin are the first principal components of the leg point group data and the shin point group data, respectively, and have the direction that most characterizes the point group data. This is because the leg and the shin each have a cylindrical shape and their longitudinal directions are used as axes. In contrast, the front direction of the knee joint is the third principal component of the knee joint point group data. The patella covering the knee joint has a disc shape, and the first principal component and the second principal component (the component that is orthogonal to the first principal component and has the largest variance) appear as components along the surface of the patella. Since the third principal component has a direction orthogonal to the first principal component and the second principal component, it becomes the front direction of the knee joint. The axis of the shin estimated by the partial point group detection unit 14 is temporarily obtained for use in the estimation of the axis of the shin by the estimation unit 16. That is, the estimation unit 16 can obtain the axis of the shin with higher accuracy than the partial point group detection unit 14.

[0015] The convex part determination unit 15 determines the convex part of the shin in the cross-sectional view included in the shin point group data obtained from the cross-section of the shin. Here, the cross-section is a plane perpendicular to the longitudinal direction among the directions in the three-dimensional space of the shin. The shin has a characteristic shape such that the tibia serving as the axis protrudes. That is, the convex part of the shin refers to the part where the tibia protrudes in the shin. N - 1 vertical equal division planes that equally divide the axis of the shin point group data are obtained, and for each of the N - 1 partial spaces sandwiched between two opposing vertical equal division planes, the points located within the partial space are projected onto a plane viewed from the axial direction of the shin point group data, thereby obtaining N - 1 cross-sectional views of the shin. The convex part determination unit 15, for example, performs circle detection by the Hough transform from the cross-sectional view and determines the point farthest from the center as the convex part. FIG. 5 is a diagram showing the method for determining the convex part by circle detection. In FIG. 5, A is the point group data composed of the points included in the cross-sectional view of the shin in the shin point group data. B is the circle detection by the Hough transform, and the point indicated by the circle 71 is the point farthest from the center and is determined as the convex part. The convex part determination unit 15 takes a plurality of cross-sectional views and determines the positions of a plurality of convex parts.

[0016] The estimation unit 16 estimates the axis of the tibia based on the points detected as convex portions. More specifically, the estimation unit 16 estimates the six degrees of freedom of the tibia based on the cross-section of the tibia. That is, the estimation unit 16 estimates the rotation angle around each axis of the tibia's orthogonal coordinate system and the displacement along each axis. The estimation unit 16 estimates the six degrees of freedom of the tibia using the following procedure. The estimation unit 16, for example, projects the points of the convex portion detected as a convex portion in each cross-sectional view onto a three-dimensional space by arranging them in the corresponding subspaces, and then estimates the axis of the tibia by determining the straight line obtained by applying the least squares method in the three-dimensional space as the axis of the tibia. In this way, the estimation unit 16 can estimate the displacement along each axis of the orthogonal coordinate system of the tibia.

[0017] Furthermore, the estimation unit 16 projects the points of the convexity into three-dimensional space by arranging the centers of multiple circles detected by the convexity determination unit 15 in multiple cross-sectional views into corresponding subspaces, and determines the straight line obtained by applying the least squares method in three-dimensional space as the central axis of the lower leg. Based on the positional relationship between the axis of the tibia and the central axis of the lower leg in three-dimensional space, the estimation unit 16 estimates the rotation of the central axis of the lower leg around the axis of the tibia. Since the tibia is located on the anterior surface of the lower leg, the rotation direction of the tibia can be determined by the rotation of the central axis of the lower leg around the axis of the tibia. The estimation unit 16 decomposes the rotation of the central axis of the lower leg around the axis of the tibia into rotation components around each axis of the Cartesian coordinate system. This allows the estimation unit 16 to estimate the rotation angle of the tibia around each axis of the Cartesian coordinate system. In other words, the estimation unit 16 can estimate the six degrees of freedom of the tibia.

[0018] The angle measuring unit 18 measures the angle between the axis of the tibia estimated by the estimation unit 16 and the axis of the femur estimated by the partial point cloud detection unit 14. Here, the angle includes the flexion / extension angle and the varus / valgus angle. The flexion / extension angle is the angle between the two axes when the knee joint is flexed and extended. The varus / valgus angle is the angle between the tibia and femur in the medial and lateral directions, and is the angle between the two axes when the flexion / extension angle of the two axes is 0 degrees. The angle measuring unit 18 may measure the angle between two axes using the axis of the tibia and the axis of the femur estimated by the partial point cloud detection unit 14.

[0019] <Knee joint template creation device> The knee joint template creation device 2 creates a knee joint template to be used by the knee joint detection unit 13. Figure 6 shows the configuration of the knee joint template creation device 2. The knee joint template creation device 2 comprises a point cloud data acquisition unit 20, a downsampling unit 21, a feature calculation unit 22, a feature recording unit 23, and a knee joint template storage unit 24.

[0020] The point cloud data acquisition unit 20 acquires point cloud data related to the knee joint. The point cloud data related to the knee joint is, for example, data obtained by photographing the leg with a depth camera, converting it into point cloud data, and then manually extracting the data by a human. Figure 7 shows an example of the extracted point cloud data related to the knee joint.

[0021] The downsampling unit 21 reduces the number of point cloud data related to the knee joint. The downsampling unit 21 reduces the number of point cloud data by discretizing the point cloud data using, for example, predetermined voxels and using the average value as a representative point.

[0022] The feature calculation unit 22 calculates the features of the downsampled point cloud data. For example, the feature calculation unit 22 performs normal estimation for all points that make up the downsampled point cloud data and calculates FPFH features that show the relationships between each point.

[0023] The feature recording unit 23 records the calculated feature quantities as a knee joint template in the knee joint template storage unit 17 of the axis estimation device 1.

[0024] With the above configuration, the axis estimation device 1 according to the first embodiment can estimate the axis of the tibia of a person captured by the depth camera 10.

[0025] Figure 8 is a flowchart showing the operation of the axis estimation device 1. First, the depth image acquisition unit 11 acquires a depth image captured by the depth camera (step S101). Then, the point cloud data conversion unit 12 converts the depth image into point cloud data (step S102). The knee joint detection unit 13 detects the knee joint point cloud data (step S103). Then, the partial point cloud detection unit 14 detects the shin point cloud data and the thigh point cloud data (step S104). The partial point cloud detection unit 14 performs principal component analysis on the point cloud data (step S105). The partial point cloud detection unit 14 determines the axes of each part of the shin and thigh (step S106). The convex part determination unit 15 acquires a cross-section from the shin point cloud data (step S107). The convex part determination unit 15 performs circle detection on the cross-section (step S108). The convex part determination unit 15 determines the position of the convex part (step S109). The estimation unit 16 estimates the axis of the tibia (step S110). The angle measuring unit 18 measures the angle between the axis of the tibia and the axis of the femur (step S111).

[0026] Figure 9 is a flowchart showing the operation of the knee joint template creation device 2. First, the leg is photographed using a depth camera (step S201). The captured image is then converted into point cloud data (step S202). The point cloud data acquisition unit 20 accepts the selection of the knee joint area (step S203) and extracts the point cloud data related to the knee joint (step S204). The downsampling unit 21 performs downsampling on the point cloud data (step S205). The feature calculation unit 22 calculates the features of the point cloud data (step S206). The feature recording unit 23 records the features as a template (step S207).

[0027] <Second Embodiment> The axis estimation device 1 according to the second embodiment estimates the position of the hip joint and the axis of the femur, in addition to the operation of the axis estimation device 1 according to the first embodiment. The knee joint detection unit 13 according to the second embodiment determines the rotation and translation of the patella. For example, the knee joint detection unit 13 determines a rotation matrix and a translation matrix that show the displacement of the knee joint point cloud data between two images. Subsequently, the estimation unit 16 according to the second embodiment calculates the rotation of the femur around its axis based on the rotation of the patella. The estimation unit 16 also estimates the position of the hip joint and the axis of the femur based on the translation of the patella. For example, by assuming that the position of the hip joint is fixed, the estimation unit 16 determines the position of the hip joint at a point equidistant from the translational trajectory of the patella, and estimates the axis of the femur as the line connecting the center of rotation and the position of the hip joint.

[0028] With the above configuration, the axis estimation device 1 according to the second embodiment can estimate the axis of the tibia, as well as the position of the hip joint and the axis of the femur. Furthermore, a more accurate model of the knee joint can be created based on the knee joint position and the estimated axes of the tibia and femur. By applying knee joint instability data to this model, a knee joint instability model can be created.

[0029] <Other Embodiments> Although one embodiment of this invention has been described in detail above with reference to the drawings, the specific configuration is not limited to that described above, and various design changes can be made without departing from the spirit of this invention.

[0030] The axis estimation device 1 in the above-described embodiment may be implemented in whole or in part by a computer. In that case, the program for implementing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed. Here, "computer system" includes the OS and peripheral hardware. Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and recording devices such as hard disks built into a computer system. In addition, "computer-readable recording medium" may include those that dynamically hold programs for a short period of time, such as communication lines when transmitting programs via networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a certain period of time, such as volatile memory inside a computer system that acts as a server or client in that case. Furthermore, the above-mentioned program may be for implementing a part of the above-mentioned function, and may also be able to implement the above-mentioned function in combination with a program already recorded in the computer system. Furthermore, the axis estimation device 1 may be implemented in whole or in part using a programmable logic device such as an FPGA (Field Programmable Gate Array). [Explanation of symbols]

[0031] 1. Axis estimation device, 10. Depth camera, 11. Depth image acquisition unit, 12. Point cloud data conversion unit, 13. Knee joint detection unit, 14. Partial point cloud detection unit, 15. Convex part determination unit, 16. Estimation unit, 17, 24. Knee joint template storage unit, 18. Angle measurement unit, 2. Knee joint template creation device, 20. Point cloud data acquisition unit, 21. Downsampling unit, 22. Feature calculation unit, 23. Feature recording unit, 61. Knee joint point cloud data, 62. Shin point cloud data, 63. Thigh point cloud data

Claims

1. A shin point cloud detection step involves detecting point cloud data relating to the shin from point cloud data relating to the leg, including the knee and shin, based on template data representing the knee joint. A convexity determination step is performed for each of the multiple cross-sectional planes perpendicular to the longitudinal direction of the shin from the point cloud data relating to the shin, by determining the convexity of the shin in the cross-sectional plane from the multiple points corresponding to the said cross-sectional plane, An estimation step of estimating the axis of the tibia of the tibia based on the multiple protrusions, A method for estimating axes having the following characteristics.

2. The template data representing the knee joint shows the feature quantities calculated from the point cloud data related to the knee joint. The axis estimation method according to claim 1.

3. The system further includes a knee joint detection unit that determines the rotational center of the movement of the patella of the leg, The estimation step estimates the position of the hip joint and the axis of the femur based on the center of rotation. Axis estimation method according to claim 1 or 2.

4. A shin point cloud detection unit detects point cloud data relating to the shin from point cloud data relating to the leg, including the knee and shin, based on template data indicating the knee joint of the leg; For each of the multiple cross-sectional planes perpendicular to the longitudinal direction of the shin among the point cloud data relating to the shin, a convexity determination unit determines the convexity of the shin in the cross-sectional plane from a plurality of points corresponding to the cross-sectional plane, An estimation unit that estimates the axis of the tibia of the tibia based on the multiple protrusions, An axis estimation device having the following features.

Images