Method for modeling a joint

Through automatic posture extraction and image registration technology, combined with additional sensor data, the cumbersome and time-consuming problem of establishing high-precision joint articulation models is solved, and efficient model establishment is achieved within the conventional workflow, improving diagnostic accuracy and data Set diversity.

CN118974774APending Publication Date: 2024-11-15KONINKLIJKE PHILIPS NV
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202380027901.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-14
Filing Date
2023-03-01
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In medical image analysis and processing, establishing dynamic articulation models is cumbersome and time-consuming, and conventional imaging sessions do not allow significant changes to the workflow, making it difficult to build a complete model with sufficient accuracy.

Method used

Process 3D surface image data through an automatic posture extraction method, combine medical image data for image registration, determine transformed data to create and update statistical articulation joint models, and record posture information with additional sensor data, allowing the establishment of articulation models within a conventional workflow .

Benefits of technology

The establishment of high-precision articulation models within routine workflows is achieved, reducing the use of imaging modalities for patients, improving the accuracy of models and the diversity of data sets, and avoiding the risk of masking anatomical features of interest in diagnosis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118974774A_ABST
    Figure CN118974774A_ABST
Patent Text Reader

Abstract

A method for modeling a joint is disclosed, the method comprising: processing first 3D surface image data of a body part in a first posture by means of an automatic posture extraction method, thereby obtaining a first articulation parameter representative of the first posture, the body part comprises a first anatomical structure and a second anatomical structure which are connected by a joint; image registration is performed between first medical image data of the body part and second medical image data of the body part in a second posture, the first medical image data being acquired concurrently with the first 3D surface image data and depicting the first anatomical structure and the second anatomical structure, performing the image registration for each of the first anatomical structure and the second anatomical structure, respectively; determining transformation data representing one or more first transforms required to register the first anatomical structure in the first medical image data to the first anatomical structure in the second medical image data, and / or one or more second transformations required to register the second anatomical structure in the first medical image data to the second anatomical structure in the second medical image data; and creating and / or updating a statistical articulation joint model based at least on the transformation data, the first articulation parameter, and a second articulation parameter representative of the second pose.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention provides a computer-implemented method for modeling a joint, a data processing system, a system including a data processing system, a computer program product, and a computer-readable medium. Background Art

[0002] Many recent developments in medical image analysis and processing require dynamic anatomical models, also known as articulated models, which represent postures such as flexion of joints or inhalation / exhalation of the chest, for example.

[0003] For example, the 3D joint model can be used for quality assessment, preparation for diagnosis and / or treatment planning. In particular, certain diagnostic problems require specific image acquisition whereby the joint flexion is within a specific range, because deviations from such flexion may result in certain anatomical features of diagnostic interest being obscured within the projection image.

[0004] The articulated model can be used to assess the positioning of the object based on image data alone and, once it is fitted to the available data, also allows to judge (mal)normality of the joint kinematics and range of motion.

[0005] Establishing those articulated models on a broad patient population is tedious and time-consuming.

[0006] Furthermore, building articulated models requires the use of imaging modalities that use, for example, ionizing radiation, which can be harmful to patients. Therefore, voluntary scans outside of regular imaging sessions, such as routine checkups of patients, may simply not be available in sufficient numbers and also pose ethical issues.

[0007] Conventional imaging sessions, such as routine examinations of patients, do not typically allow for significant workflow changes, especially in cases where they are time consuming, for example, imaging a joint in multiple postures using the imaging modality used for the imaging session. Clinical workflows typically do not allow for the addition of steps that slow down routine scheduling and throughput, such as performing dedicated measurements for different postures during an examination. For example, externally measuring flexion at different postures for multiple images obtained by the imaging modality of the workflow and recording them to be stored with the images for co-registration and sorting according to flexion angles, and building models based on this, is difficult to incorporate into conventional workflows.

[0008] Therefore, it may be difficult to build a complete model with sufficient accuracy. Summary of the invention

[0009] It is an object of the invention to allow improved modelling of joints.

[0010] According to the independent claims, the invention provides a method for modeling a joint, a data processing system, a system comprising a data processing system, a computer program product and a computer readable medium.

[0011] A method for modeling a joint according to the present disclosure includes: processing first 3D surface image data of a body part in a first posture by means of an automatic posture extraction method to obtain first articulation parameters representing the first posture, wherein the body part includes a first anatomical structure and a second anatomical structure connected by a joint.

[0012] The method for modeling a joint according to the present disclosure also includes: performing image registration between first medical image data of the body part and second medical image data of the body part in a second posture, the first medical image data being acquired simultaneously with the first 3D surface image data and depicting the first anatomical structure and the second anatomical structure. The image registration is performed separately for each of the first anatomical structure and the second anatomical structure.

[0013] The method for modeling a joint according to the present disclosure also includes determining transformation data, which represents one or more first transformations required to align a first anatomical structure in a first medical image data to a first anatomical structure in a second medical image data and / or one or more second transformations required to align a second anatomical structure in the first medical image data to a second anatomical structure in the second medical image data.

[0014] The method for modeling a joint according to the present disclosure further includes: creating and / or updating a statistical articulation joint model based on at least the transformation data, the first articulation parameter, and a second articulation parameter representing a second posture.

[0015] In other words, the limitations of known methods can be addressed by relying on additional sensor data (in addition to data from the imaging modality) that provides depth information (e.g. from a surface scanner or a rangefinder camera or other 3D camera system, such as an RGBD camera) to record posture information during an examination using an imaging modality. This enables correlation of posture with images from the imaging modality acquired in a conventional workflow. Based on this correlation, an articulated model can be built within the standard workflow of a radiology department.

[0016] Specifically, the disclosed method allows arbitrary postures to be acquired without the need for additional recorded information. For example, the user is not required to observe the articulated posture for a given acquired joint, and the posture data does not have to be manually measured / tracked by a radiologist. This is particularly useful when measuring posture parameters using measurement equipment would be a burden for the imaging modality, such as due to space limitations within a CT chamber, or due to the inability to use metal in an MRI scanner.

[0017] Therefore, since the proposed method does not require interruption of the imaging workflow, it allows images from the clinical routine workflow to be included into the data used to build the articulated model. Thus, large and diverse datasets can be used to build the model.

[0018] As an example, the present disclosure can be used for the development of an articulated joint model, where the model is created using MR scans of the joint, where the joint has been pre-positioned in one or more known postures or articulations.

[0019] A potential application of the articulated model is in bone X-ray image quality assessment.

[0020] As can be seen from the above, the claims require the creation and / or updating of an articulation model. For example, the method requires the use of any articulation model that may be empirical, semi-empirical or theoretical as a starting point, and the application of the steps of the present disclosure, particularly repeated for one or more objects and / or one or more imaging sessions of the same object or different objects, to improve the model. Improving the model may involve different aspects. For example, the model may become less generic and more closely represent reality, particularly statistically over a wide range of objects. In addition, the accuracy of the model can be improved, for example, by allowing the size of the data set to be meaningfully increased. In addition, the range of the model can be extended toward the end of the movement spectrum (e.g., postures representing very strong flexion and / or postures representing hyperextension of the joint) relative to the extreme postures.

[0021] In this disclosure, except for the description of the related art, the term "joint model" is used synonymously with the term "statistical articulation joint model" for the purpose of readability.

[0022] Joints are the connections that movably connect bones in the body.

[0023] For example, a body part may be an arm, wrist, shoulder, ankle, leg, hip, foot, hand, or any other part of the body that includes a joint.

[0024] In the present disclosure, the term "anatomical structure" refers to a subsurface portion of the body. For example, an anatomical structure may be a bone, a muscle, a cartilage, a ligament, etc. It should be noted that different anatomical structures may have different degrees of rigidity, so that the movement of some anatomical structures may have more degrees of freedom than the movement of other anatomical structures. For example, bones are rigid and incompressible in nature, so that their movement can generally be represented by translation and rotation. Muscles or cartilage may deform elastically, for example, lengthen, shrink or compress, so their movement may also include (particularly elastic) deformation and abrasion.

[0025] A pose is a certain relative arrangement of a set of anatomical structures. For example, a pose may be defined by the relative arrangement of two bones, e.g., one or more angles describing their relative arrangement in one or more degrees of freedom.

[0026] The posture can be represented by an articulation parameter. The articulation parameter can be any parameterization of the relative arrangement of the anatomical structure. For example, the articulation parameter can include a flexion angle.

[0027] It should be understood that the first posture and the second posture are different, for example, they differ in at least one articulation parameter. For example, the first posture and the second posture may differ in that they have different flexion angles.

[0028] Automatic pose extraction can be any method known in the art, for example from gaming applications, which determines the pose of a body part based on 3D surface image data. The 3D information allows resolving directional ambiguities that might not be resolved otherwise.

[0029] It should be noted that according to claim 1, second articulation parameters representing the second posture are used. The second articulation parameters may be data derived from second 3D surface image data of the body part obtained using automatic posture extraction. Alternatively, the second articulation parameters may be articulation parameters derived from second medical image data and / or image data obtained using a different imaging modality and / or image data obtained by direct posture measurement.

[0030] 3D surface image data is data providing depth information. For example, the 3D surface image data may be obtained, for example, by a 3D surface scanner, a stereo camera and / or an RGBD camera.

[0031] In the present disclosure, medical image data refers to image data depicting subsurface structures of the body, for example, image data obtained using ionizing radiation. For example, medical image data may include CT image data, MRI data, or X-ray image data. Medical image data may include 2D and / or 3D image data. 3D image data provides depth information of the imaged structures. For example, 2D image data may depict projections of the imaged structures.

[0032] According to the present disclosure, first 3D surface image data and first medical image data are collected simultaneously. This may include, during an imaging session, continuously collecting the first 3D surface image data within a time period including one or more times when the first medical image data is collected. If there is no corresponding medical image data, for example, when it is obtained in the time between capturing medical images, the 3D surface image data (in particular, the 3D surface image data from the time period) can also be used to build the model. For example, such image data can be used to improve the accuracy of automatic posture extraction or to improve the accuracy when building a joint model, for example, by inferring key points of the anatomical structure from the surface image data, as explained in further detail below, or by helping to eliminate ambiguity.

[0033] Image registration may be performed using registration methods known in the art, including: 2D / 3D registration, where 2D image data is registered with 3D image data and / or a mesh; and 3D / 3D registration, where 3D image data is registered with 3D image data and / or a mesh.

[0034] According to the present disclosure, image registration is performed for each anatomical structure separately. This means that the first anatomical structure and the second anatomical structure are independently registered. Nevertheless, image registration can still be performed for the first anatomical structure and the second anatomical structure in parallel.

[0035] Transformations according to the present disclosure may include rotations and / or translations. In addition, transformations may include deformation and scaling, for example, where the anatomical structure is non-rigid and / or compressible.

[0036] An articulation joint model may be a model that represents the movement of a joint (particularly the anatomical structure of the joint) between different articulations.

[0037] The articulated joint model can be: a 2D model, which models the joint in projection; or, a 3D model, which has depth information of the joint (particularly the depth information of the anatomical structure). For the sake of completeness, even if the articulated joint model is a 2D model, the process of creating and / or updating the model involves some kind of 3D data to allow ambiguity to be avoided.

[0038] The statistical model represents data obtained from multiple imaging sessions (particularly more than one subject). As an example, the same joint may be imaged for multiple subjects, and a model may be built to average out differences between the different subjects, e.g., differences in size and shape, differences in alignment of anatomical structures, etc. The statistical model may also be selectively based on data from multiple subjects that share certain characteristics, as described in more detail below.

[0039] Creating a statistical articulated joint model may require starting from a model in which no overall joint exists, for example starting from individual models of a first anatomical structure and a second anatomical structure, and / or starting from a rigid model of the joints and / or a non-statistical model of the joints (articulated or non-articulated) to obtain a statistical articulated joint model.

[0040] Updating the statistical articulation joint model may comprise modifying a current version of the statistical articulation joint model to take into account the additional information, in particular the transformation data, the first articulation parameters and the second articulation parameters representing the second pose.

[0041] Thus, creating and / or updating a statistical articulation joint model based on the transformation data, the first articulation parameters and the second articulation parameters representing the second pose may be seen as taking into account said data and parameters to model the joint, optionally together with existing data for modeling the joint.

[0042] According to the present disclosure, a statistical articulation joint model may be based on transformation data, first articulation parameters, and second articulation parameters from image data acquired in a plurality of imaging sessions of the subject and / or from one or more imaging sessions for each of a plurality of subjects.

[0043] The imaging session may be a routine examination and in particular may not include changing the routine workflow, in particular may not include acquiring medical image data beyond that which is part of the routine examination, and may not include performing manual measurements of the posture. By allowing data from different imaging sessions of the same or different subjects to be combined, large data sets may be processed and accurate statistics performed accordingly, allowing for more accurate models.

[0044] Alternatively or additionally, according to the present disclosure, creating and / or updating a statistical articulation joint model may include determining a mean over a randomly selected subset of the population and / or determining a mean over a subset of the population of types of misalignment of shared joints.

[0045] When using data collected from multiple (particularly randomly selected) subjects, a statistical model of average subject-specific characteristics can be provided.

[0046] When only data collected for a subset of the population that shares the type of misalignment is used for the statistical model, this can allow the characteristics of the joints associated with the misalignment to be determined. This can help to better address the needs brought about by the misalignment and still have a large data set compared to just a single subject.

[0047] According to the present disclosure, the statistical articulation joint model can be a 3D model. For example, a 3D surface of each anatomical structure of the joint can be provided, especially over the entire range of the model. This allows for a particularly detailed study of the anatomical structures and their interactions.

[0048] The method of the present disclosure may include: for each of a plurality of postures including a first posture P1 and a second posture P2, creating a constellation model C (φ0, . . . , φ n ) in order to obtain a plurality of constellation models, said constellation models representing the arrangement of the first anatomical structure and the arrangement of the second anatomical structure as a function of an articulation parameter representing the corresponding posture, in particular as a function of the flexion angle of the joint. The constellation model can thus be used as input data for creating and / or updating a statistical articulation joint model. The arrangement can be determined relative to any suitable common reference.

[0049] According to the present disclosure, creating and / or updating a statistical articulated joint model may be based on a plurality of constellation models, and in particular, may include combining a plurality of constellation models. For example, a constellation model may be used to sort medical images in an image space to create a joint model. In addition, a constellation model may be used together with interpolation to obtain an articulated model including poses for which no image data is available.

[0050] According to the present disclosure, image registration may include applying a registration algorithm that registers image data, such as based on landmarks in the image data and / or a mesh obtained by a segmentation algorithm, such as based on one or more vertices of the mesh. Specifically, the method of the present disclosure may include: before image registration, applying a segmentation algorithm to segment the first anatomical structure and / or the second anatomical structure to obtain a mesh. Any known segmentation and / or image registration method may be applied. Anatomical atlas data may be used.

[0051] The method of the present disclosure may include performing pre-registration using the first articulation parameter and the second articulation parameter.

[0052] For example, the first articulation parameter and the second articulation parameter may be used to perform a transformation that roughly matches the anatomical structure before implementing an image registration algorithm. This would reduce the resources required for image registration and would also resolve ambiguities that image registration could not resolve.

[0053] According to the present disclosure, a first anatomical structure may be used as a reference anatomical structure, and a statistical articulation joint model may represent the arrangement of a second anatomical structure relative to the first anatomical structure.

[0054] For example, the flexion angle may be expressed relative to a reference anatomical structure. The reference anatomical structure may be one of the one or more bones of the joint.

[0055] According to the present disclosure, the statistical articulation joint model may be a model representing one or more selected movement types, and the method may include: filtering data representing one or more selected postures of the one or more selected movement types (in particular, one or more selected articulation parameters) so that only data representing one or more selected movement types are used to create and / or update the statistical articulation joint model.

[0056] An articulated joint model representing (one or more) selected movement types facilitates a detailed characterization of the joint. It can also reduce unnecessary complexity associated with only one movement type for a given application.

[0057] For example, a movement type may be a rotation and / or bending only in a first plane, or a bending only in a second plane. A movement may be represented by a set of postures that are part of the movement. For example, a movement type may be represented by only a subset of all possible postures of a joint. For example, a movement type may be represented by only postures that have the same flexion angle in one direction and different flexion angles in another direction. Thus, the articulation parameters representing a selected movement type may be the articulation parameters corresponding to the selected posture representing the selected movement type.

[0058] Filtering the data for the selected pose may entail using only the data (eg, image data and / or mesh) corresponding to the selected pose as input for creating the articulation joint model.

[0059] The method of the present disclosure may include: anatomical structure-by-anatomical structure registration of key points inferred from 3D surface image data of a body part to transform a reference segmentation of a first anatomical structure and / or a second anatomical structure from a first posture P1 to a third posture P3; and, creating and / or updating a statistical articulation joint model based on articulation parameters of the third posture P3 and the transformed reference segmentation and / or corresponding transformation data.

[0060] As an example, bones or other anatomical structures may have an effect on the surface of the object, for example, which may be discernible through the skin and / or tissue. For example, protrusions may be formed on the surface. Points at which anatomical structures are discernible on the surface may be detected in the surface image data. The positions of the detected points may then allow estimating the positions of one or more key points of the anatomical structure (e.g., on a mesh), specifically based on 3D image data and medical image data in a given posture optionally supplemented by general anatomical data. On this basis, for other postures for which there is no available medical image data, detecting points at which anatomical structures are discernible on the surface in the 3D image data allows the positions of key points to be inferred. On this basis, the transformation of key points from one posture to another can be determined, and the mesh may also be transformed based on the transformation of the key points. Thus, even for postures for which only surface image data is available, additional data of the anatomical structure may be obtained. Examples are described in more detail below.

[0061] The method of the present disclosure may include determining, in first 3D surface image data of the body part, one or more key points of a surface of the body part in the first 3D surface image data corresponding to key points of a first anatomical structure and / or a second anatomical structure in the first medical image data.

[0062] A keypoint of a surface may be a point on the surface of an object where a portion of an anatomical structure is discernible, for example by creating a protrusion on the surface. A corresponding keypoint of an anatomical structure may be a point of the surface of a portion of an anatomical structure that is discernible on the surface, for example forming a protrusion.

[0063] The method disclosed herein may further include determining the position of each of one or more key points of the surface of the body part, and the position of each of one or more key points of the first anatomical structure and / or the second anatomical structure in the first posture P1, for example, with the aid of a reference segmentation of the first anatomical structure / the reference segmentation and / or a reference segmentation of the second anatomical structure / the reference segmentation.

[0064] The method of the present disclosure may further include: processing third 3D surface image data of the body part arranged in a third posture P3 to identify the positions of one or more key points in the surface of the body part in the third posture P3. It should be understood that the third posture may be different from the second posture and different from the first posture.

[0065] The method disclosed herein may further include determining the position of each key point of the corresponding key points of the first anatomical structure and / or the second anatomical structure in the third posture P3 based on the position of each key point of one or more key points of the surface of the body part in the third posture P3, the position of each key point of one or more key points of the surface of the body part in the first posture P1, and the position of each key point of one or more key points of the first anatomical structure and / or the second anatomical structure in the first posture P1.

[0066] The method disclosed herein may further include determining key point transformation data, wherein the key point transformation data represents one or more third transformations required to match each key point of one or more key points of the first anatomical structure and / or the second anatomical structure in the first posture P1 with each key point of the first anatomical structure and / or the second anatomical structure in a third posture P3.

[0067] The method of the present disclosure may further include: creating and / or updating a statistical articulation joint model based on the key point transformation data and a third articulation parameter representing the third posture P3.

[0068] Thus, surface image data for poses for which no available medical image data exists (e.g., the third pose) can be used as input to the articulated model. This allows for an even wider set of input data for the model, which also improves the accuracy of the model.

[0069] The method of the present disclosure may include the following steps: when transforming from the first posture P1 to the third posture P3, checking for potential conflicts between the reference segmentation of the first anatomical structure and the reference segmentation of the second anatomical structure, and restricting the movement of the anatomical structures accordingly.

[0070] This allows avoiding extrapolation errors and / or other modeling errors. In particular, where the real anatomical structures do not collide, the parts of the articulated model that represent the collision will be inaccurate and can therefore be discarded.

[0071] The method of the present disclosure may include simultaneously acquiring first 3D surface image data and first medical image data. That is, the method may include a step of data acquisition for creating and / or updating data of the model. This may be accomplished by corresponding 3D surface imaging means and medical imaging means, for example, as described above.

[0072] The present invention also provides a data processing system to perform and / or control any one of the method steps of the present invention. Specifically, the data processing system can be configured to perform one or more and specifically all of the calculation steps of the present invention, and / or control the acquisition of 3D surface image data and / or medical image data, for example by controlling a 3D surface imaging system configured to acquire 3D surface image data and / or an imaging system configured to acquire medical image data.

[0073] Specifically, the data processing system may include one or more processors, in particular one or more computing devices and / or a distributed network of computing devices including the one or more processors, and the one or more processors may be configured to execute and / or control the steps of the method of the present disclosure.

[0074] The present invention also provides a system, comprising the data processing system of the present disclosure, and further comprising a 3D surface imaging system, wherein the 3D surface imaging system is configured to acquire 3D surface image data, wherein the 3D surface image data comprises first 3D surface image data and / or second 3D surface image data and / or third 3D surface image data. The system further comprises an imaging system, wherein the imaging system is configured to acquire medical image data (including first medical image data and / or second medical image data), in particular a CT imaging system and / or an MRT imaging system and / or an X-ray imaging system. The system can be configured to perform any one of the methods of the present disclosure.

[0075] The present invention also provides a computer program product comprising instructions, which, when the program is executed by a computer, cause the computer to perform and / or control any method steps described in the present disclosure.

[0076] The present invention also provides a computer-readable medium comprising instructions, which, when executed by a computer, cause the computer to perform and / or control any one of the method steps described in the present disclosure.

[0077] The present disclosure also provides for the use of a statistical articulation joint model for at least one of: quality assurance of measurement data, correct placement / arrangement of objects, correct measurement of objects, stimulation and / or visualization of medical imaging devices, checking for object-specific articulations, deviations and limitations.

[0078] The features and advantages outlined in the context of the method for modeling a joint also apply to the data processing system, including the data processing system, computer program product and computer-readable medium of the present disclosure.

[0079] Additional features, examples and advantages will become apparent from the detailed description taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 shows a schematic diagram of a system according to the present disclosure;

[0081] Figure 2a and 2b A schematic non-scale representation of two different types of joints is shown;

[0082] Figure 3 A flow chart representing a method according to the present disclosure is shown;

[0083] Figure 4a and 4b Different views showing different types of joints;

[0084] Figures 5a to 5c shows an example of a 3D articulated model and a knee segment view; and

[0085] Figure 6 Another exemplary flow chart of a method according to the present disclosure is shown.

[0086] List of reference numerals:

[0087] System 1

[0088] Data processing system 2

[0089] 3D surface imaging system3

[0090] Imaging system 4

[0091] Body Part 5

[0092] Joint 6

[0093] First Anatomy 7

[0094] Second anatomical structure 8f

[0095] Third Anatomical Structure 9

[0096] Data storage device 10

[0097] Data Connection 11

[0098] First pose P1

[0099] Second pose P2

[0100] The third posture P3

[0101] Fourth posture P4

[0102] Flexion angle φ DETAILED DESCRIPTION

[0103] Figure 1 A schematic representation of a system according to the present disclosure is illustrated.

[0104] The system 1 comprises a data processing system 2 according to the present disclosure. As shown in the present example, the system may also comprise a 3D surface imaging system 3 (e.g. a depth camera) and an imaging system 4 configured to acquire 2D and / or 3D medical imaging data, e.g. an MRI or CT scanner or an X-ray imaging device. The data processing system 2 may comprise one or more processors, in particular one or more computing devices and / or a distributed network of computing devices comprising one or more processors.

[0105] The system 1 is configured to perform the method of the present disclosure, such as the method specified in the method claim or as described below in Figure 3 and 4. In particular, the data processing system may be configured to perform and / or control the method of the present disclosure, such as the method specified in the method claims or as described below.

[0106] Specifically, the data processing system may be configured to process first 3D surface image data of a body part in a first posture by means of an automatic posture extraction method, thereby obtaining first articulation parameters representing the first posture. Figure 2a and 2b , and are described separately in their context (e.g., a leg and an ankle). A body part includes a first anatomical structure and a second anatomical structure connected by a joint, such as in Figure 2a and 2b The system may also be configured to perform image registration between first medical image data of a body part and second medical image data of the body part in a second posture. The first medical image data (e.g., CT, MRI, or X-ray image data) is image data acquired simultaneously with the first 3D surface image data, and depicts a first anatomical structure and a second anatomical structure. The system is configured to perform image registration for each of the first anatomical structure and the second anatomical structure, respectively. The system may also be configured to: determine transformation data representing one or more first transformations required to register the first anatomical structure in the first medical image data to the first anatomical structure in the second medical image data, and / or one or more second transformations required to register the second anatomical structure in the first medical image data to the second anatomical structure in the second medical image data; and, create and / or update a statistical articulation joint model based at least on the transformation data, the first articulation parameters, and the second articulation parameters representing the second posture. If the articulation joint model is updated, the system may be configured to access a stored current statistical articulation joint model (e.g., locally or remotely) and update the accessed model. The system may then store the created and / or updated models (eg, locally or remotely).

[0107] The 3D surface imaging system 3 may be configured to acquire 3D surface image data as used in the method claims or as described below. The medical imaging system may be configured to acquire medical image data, for example CT, MRI or X-ray image data as used in the method claims or as described below.

[0108] In addition to being configured to perform the above steps, the data processing system may be configured to control image acquisition with the aid of a 3D surface imaging system and / or a medical imaging system, thereby obtaining corresponding image data. Alternatively or additionally, the processing system may be configured to retrieve previously stored image data and / or models, for example, stored locally and / or remotely in one or more data storage devices 10. The data processing system may be connected to (one or more) imaging systems and / or (one or more) data storage systems, wherein the image data and / or models are stored via a data connection 11 (e.g., a wireless and / or wired data connection).

[0109] Figure 2a and 2b are schematic, non-scale representations of two different types of joints.

[0110] exist Figure 2a In the figure, as an example, the body part 5 is a lower limb of a human subject, and the knee is shown as an example of a joint 6. The first anatomical structure 7 can be a femur, and the second anatomical structure 8 can be a tibia. In addition, the patella is shown as a third anatomical structure 9.

[0111] exist Figure 2b , as an example, the body part is the ankle of a human subject, and the ankle joint is shown as an example of joint 6. The tibia, talus, and calcaneus are examples of first anatomical structure 7, second anatomical structure 8, and third anatomical structure 9. It should be noted that the knee has only one degree of freedom, and therefore, there is only one flexion angle for each posture. The ankle joint has two independent degrees of freedom, and therefore, there are two flexion angles for each posture.

[0112] As mentioned above, in the general part of the description, the system and method of the present disclosure can provide a constellation model, which is a collection of flexion angles for each of a plurality of postures. The constellation model can then be used to obtain an articulation joint model.

[0113] exist Figure 3 , a flowchart illustrates an exemplary method for modeling a joint according to the present disclosure.

[0114] In step S11, first 3D surface image data of a body part in a first posture is processed by an automatic posture extraction method to obtain first articulation parameters representing the first posture. The body part comprises a first anatomical structure and a second anatomical structure connected by a joint.

[0115] In step S12, image registration is performed between first medical image data of the body part and second medical image data of the body part in a second posture. The first medical image data is acquired simultaneously with the first 3D surface image data and depicts a first anatomical structure and a second anatomical structure. The image registration is performed for each of the first anatomical structure and the second anatomical structure, respectively.

[0116] In this example, the medical image data may be 3D medical image data, such as 3D CT or MRI image data. Therefore, in this case, 2D / 3D registration of the medical image data onto a mesh may not be required to obtain 3D information.

[0117] In step S13, transformation data is determined, wherein the transformation data represents one or more first transformations required to align the first anatomical structure in the first medical image data to the first anatomical structure in the second medical image data, and / or one or more second transformations required to align the second anatomical structure in the first medical image data to the second anatomical structure in the second medical image data.

[0118] In step S14, a statistical articulation joint model is created and / or updated based on at least the transformation data, the first articulation parameters and second articulation parameters representing a second posture.

[0119] exist Figure 3 , optional steps S15 and S16 are shown. In step S15, an anatomical structure level (e.g., bone level) registration of key points of anatomical structures in the medical image data inferred from the 3D surface image data of the body part may be performed. For example, the shape of the surface of the body part may indicate the shape and position of bones or other anatomical structures beneath the surface, so that the positions of key points of corresponding anatomical structures in the medical image data may be estimated using detection of these shapes in the 3D surface image. The estimated positions of the key points are used to transform a reference segmentation of the first anatomical structure and / or the second anatomical structure from a first pose to a third pose.

[0120] In step S16, the articulation joint model is updated based on the articulation parameters of the third pose and the transformed reference segmentation and / or the corresponding transformation data. It should be noted that steps S15 and S16 can be performed after or after steps S11 to S14. If they are performed before steps S11 to S14, they can be used to create a joint model, which can then be updated in step S14.

[0121] Optional steps S15 and S16 allow incorporation of information derived from the 3D surface image data into the joint model in the absence of corresponding medical image data available. This may be particularly useful when the 3D surface image data within one imaging session is acquired at a higher rate than the medical image data.

[0122] It should be noted that the method may optionally comprise a step S10 of simultaneously acquiring 3D surface image data and medical image data. However, alternatively, the image data may also be at least partly extracted image data.

[0123] Hereinafter, an exemplary method for modeling a joint according to the present disclosure will be described, wherein the medical image comprises a 2D medical image. For example, most steps may be similar to Figure 3 Same as described above.

[0124] However, an additional step S11a is performed, wherein a 2D / 3D registration of the 2D first medical image data and the second medical image data is performed before the registration step S12. For example, the 2D medical image data may be registered to a common grid representing the 3D shape of the surface of the corresponding anatomical structure, as will be described in more detail below.

[0125] The registration S12 and the transformation step S13 may be performed using the 2D / 3D registered medical image data.

[0126] As an example, the above method can be applied where a medical imaging modality is used that depicts anatomical structures (such as bones) in 2D projections, such as a conventional X-ray imaging system. As described above, 2D medical image data can be used in a method according to the present disclosure to create and / or update an articulated joint model by 2D / 3D registering the first 2D medical image data and the second 2D medical image data to a common mesh topology. Thus, the 2D image data can be accurately registered and incorporated into the articulated joint model. Specifically, the 2D image data can also be used to create and / or update a 3D articulated joint model by performing a 2D / 3D registration.

[0127] The mesh may be obtained from a pre-existing 3D model of each of the anatomical structures, such as a pre-existing 3D articulated model of the joints and / or a pre-existing rigid model of the joints or individual anatomical structures.

[0128] Preferably, the same grid is used when repeating the 2D / 3D registration for additional image data to be incorporated into the joint model. As an example, since the joint model is a statistical model, it may incorporate data from several different subjects, so an average grid of these subjects may be employed, which may be updated with each new additional medical image data set. The initial grid may be created manually or semi-automatically or automatically.

[0129] 2D / 3D registration of anatomical structures using a grid may be performed using methods known in the art.

[0130] Figure 4a and 4b The diagram shows different joints in different postures P1 to P4. Figure 4a In the case of Figure 4b In this case, the ankle.

[0131] exist Figure 4a and 4b , the flexion angle φ is indicated for each posture. The knee has only one degree of freedom and, therefore, only one flexion angle for each posture. The ankle joint has two independent degrees of freedom and, therefore, there are two flexion angles for each posture. However, for simplicity, only the flexion of the upper ankle joint is shown, i.e., dorsiflexion and plantar flexion, i.e., the flexion angle of the lower ankle joint (i.e., the inversion angle or valgus angle) remains unchanged.

[0132] Figure 5a and 5b An example of a 3D articulated model of the knee is shown. Figure 5c A mesh is shown, i.e. a segmented view of the knee, which can be used for different purposes to create and / or update a joint model, as described above.

[0133] Figure 6 Another exemplary flow chart of the method according to the present disclosure is shown. The medical imaging modality and the depth camera are used to simultaneously record a medical image and a depth image of an object. Figure 6 This information is processed in a dedicated processing pipeline as shown in order to correctly co-register the medical images and generate or refine the articulated model.

[0134] The building blocks of the exemplary method are:

[0135] A 3D camera or a range-finding camera that records the object during image formation.

[0136] A pose extraction algorithm for the joint of interest, such as known from computer games, in particular deriving the flexion angle(s) φ 0..n .

[0137] A segmentation algorithm used to segment bone(s) within an image.

[0138] Registration algorithms that register images or meshes based on the corresponding skeletal poses measured using a previous (reference) skeleton.

[0139] A registration algorithm that uses the measured corresponding skeleton poses to register all other bones.

[0140] A translation / rotation transform that describes the relative movement of each bone.

[0141] Output: Constellation model C(φ0,...,φ n ), which describes the position of the model parts for a given pose.

[0142] The above describes the construction of a fully automatic joint model based on (depth camera signal, medical image) pairs.

[0143] The registration in steps 4 and 5 can also be based on the segmented skeleton, or on both the segmented skeleton and the estimated pose. In both cases, the pose estimation based on the depth signal can be used to measure the articulation parameters and thus to sort the cases. Alternatively, the articulation parameters can be derived from the rotational degrees of freedom of the registration transform (encoding the rotation about the reference pose).

[0144] Depth-based pose estimation can be used to filter the dataset for a specific articulation, for example to select cases where only the flexion angle of the ankle joint varies while the inversion or eversion angles are fixed to a small fixed range.

[0145] Instead of segmenting the skeleton for each image of the training queue, some reference skeleton segmentations can also be used to build the articulated skeleton model. In this case, these reference segmentations (e.g., in the form of meshes) can be transformed into other poses by performing a bone-by-bone registration of the key points inferred from the depth signal. When transforming the pose into a different pose, the algorithm can check for potential conflicts of the skeleton mesh and can then restrict the movement of the skeleton accordingly.

[0146] Although the present invention has been illustrated and described in detail in the drawings and the above description, such illustration and description should be considered as illustrative rather than restrictive. The present invention is not limited to the disclosed embodiments. In view of the above description and the drawings, it will be apparent to those skilled in the art that various modifications may be made within the scope of the present invention.

Claims

1. A computer-implemented method for modeling a joint (6), the method comprising: Processing (S11) first 3D surface image data of a body part (5) in a first posture (P1) by means of an automatic posture extraction method, thereby obtaining first articulation parameters representing the first posture (P1), wherein the body part (5) comprises a first anatomical structure (7) and a second anatomical structure (8) connected by a joint (6); performing (S12) image registration between first medical image data of the body part (5) and second medical image data of the body part (5) in a second posture (P2), the first medical image data being acquired simultaneously with the first 3D surface image data and depicting the first anatomical structure (7) and the second anatomical structure (8), wherein the image registration is performed for each of the first anatomical structure (7) and the second anatomical structure (8), respectively; determining (S13) transformation data representing one or more first transformations required to register the first anatomical structure (7) in the first medical image data to the first anatomical structure (7) in the second medical image data, and / or one or more second transformations required to register the second anatomical structure (8) in the first medical image data to the second anatomical structure (8) in the second medical image data; and A statistical articulation joint model is created and / or updated (S14) based at least on the transformation data, the first articulation parameters and second articulation parameters representing the second pose (P2).

2. The method according to claim 1, in, The statistical articulation joint model is based on transformation data from image data acquired in a plurality of imaging sessions for the subject and / or from one or more imaging sessions for each of the plurality of subjects, a first articulation parameter and a second articulation parameter, and / or Wherein, creating and / or updating the statistical articulated joint model comprises: determining a mean on a randomly selected population subset; and / or determining a mean on a population subset of misaligned types that share the joint (6).

3. The method according to claim 1 or 2, wherein: The statistical articulation joint model is a 3D model.

4. A method according to any one of the preceding claims, include: For each of the plurality of postures including the first posture (P1) and the second posture (P2), a constellation model C (φ0, ..., φ n ) in order to obtain a plurality of constellation models, wherein the constellation models represent the arrangement of the first anatomical structure (7) and the arrangement of the second anatomical structure (8) as a function of the articulation parameters representing the corresponding postures, in particular as a function of the flexion angle (φ) of the joint (6) i ), Therein, creating and / or updating the statistical articulated joint model is based on the plurality of constellation models, and in particular comprises combining the plurality of constellation models.

5. A method according to any one of the preceding claims, in, The image registration comprises applying a registration algorithm, which registers the image data and / or the grid obtained by the segmentation algorithm. Specifically, the method comprises: before the image registration, applying a segmentation algorithm, which segments the first anatomical structure (7) and / or the second anatomical structure (8) to obtain a grid, and / or Therein, the method comprises performing a pre-registration using the first articulation parameter and the second articulation parameter.

6. A method according to any one of the preceding claims, wherein: The first anatomical structure (7) is used as a reference anatomical structure and the statistical articulated joint model represents the arrangement of the second anatomical structure (8) relative to the first anatomical structure (7).

7. A method according to any one of the preceding claims, wherein: The statistical articulation joint model is a model representing one or more selected movement types, and the method comprises: filtering data representing one or more selected postures of the one or more selected movement types, in particular, filtering data of one or more selected articulation parameters, so that only data representing the one or more selected movement types are used to create and / or update the statistical articulation joint model.

8. The method according to any one of the preceding claims, further comprising: performing anatomy-by-anatomy registration (S15) of key points inferred from the 3D surface image data of the body part (5) to transform a reference segmentation of the first anatomical structure (7) and / or the second anatomical structure (8) from the first pose (P1) to a third pose (P3); and The statistical articulation joint model is created and / or updated (S16) based on the articulation parameters of the third pose (P3) and the transformed reference segmentation and / or the corresponding transformation data.

9. A method according to any one of the preceding claims, comprising: determining, in the first 3D surface image data of the body part (5), one or more key points of the surface of the body part (5) in the first 3D surface image data, the one or more key points corresponding to key points of the first anatomical structure (7) and / or the second anatomical structure (8) in the first medical image data; Determining the position of each of the one or more key points of the surface of the body part (5) and the position of each of the one or more key points of the first anatomical structure (7) and / or the second anatomical structure (8) in the first posture (P1), for example with the aid of a reference segmentation / the reference segmentation of the first anatomical structure (7) and / or a reference segmentation / the reference segmentation of the second anatomical structure (8); processing third 3D surface image data of the body part (5) arranged in a third posture (P3) to identify a position of each of the one or more key points of the surface of the body part (5) in the third posture (P3); Determining the position of each key point of the corresponding key points of the first anatomical structure (7) and / or the second anatomical structure (8) in the third posture (P3) based on the position of each key point of the one or more key points of the surface of the body part (5) in the third posture (P3), the position of each key point of the one or more key points of the surface of the body part (5) in the first posture (P1), and the position of each key point of the one or more key points of the first anatomical structure (7) and / or the second anatomical structure (8) in the first posture (P1); determining key point transformation data representing one or more third transformations required to match each of the one or more key points of the first anatomical structure (7) and / or the second anatomical structure (8) in the first pose (P1) with each of the key points of the first anatomical structure (7) and / or the second anatomical structure (8) in the third pose (P3); and The statistical articulation joint model is created and / or updated (S15) based on the key point transformation data and third articulation parameters representing the third pose (P3).

10. The method according to claim 8 or claim 9, comprising the following steps: When transitioning from the first pose (P1) to the third pose (P3), the reference segmentation of the first anatomical structure (7) is checked for potential conflicts with the reference segmentation of the second anatomical structure (8), and movement of the anatomical structures is restricted accordingly.

11. The method according to any of the preceding claims, comprising acquiring (S10) the first 3D surface image data and the first medical image data simultaneously.

12. A data processing system (2) configured to perform the steps of the method according to any one of claims 1 to 10 and / or to perform and / or control the steps of the method according to claim 11.

13. A system (1) comprising a data processing system (2) according to claim 12, and further comprising: a 3D surface imaging system (3), configured to acquire 3D surface image data, including the first 3D surface image data and / or the second 3D surface image data and / or the third 3D surface image data; as well as an imaging system (4), which is configured to acquire medical image data, including the first medical image data and / or the second medical image data, wherein the imaging system is specifically a CT imaging system and / or an MRT imaging system and / or an X-ray imaging system, Specifically, the system (1) is configured to perform the method according to any one of claims 1 to 11.

14. A computer program product comprising instructions which, when said program is run by a computer, cause said computer to perform the method according to any one of claims 1 to 10 and / or to perform and / or control the steps of the method according to claim 11.

15. A computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 10 and / or to perform and / or control the steps of the method according to claim 11.

Citation Information

Cited By

  • Knee joint personalized prosthesis appearance and implantation direction modulation method and system based on anatomy-motion-dynamics contradiction resolution

    CN121096669A