Computer-implemented method for providing at least one 2D unfolded image dataset

Abstract

This invention relates to a computer-implemented method for providing at least one 2D unfolded image dataset. The method includes the steps of: providing a first 3D image dataset including an anatomical structure; determining a set of representative structural points of the anatomical structure based on the first 3D image dataset; providing a 2D sample map of the anatomical structure; determining a set of representative sample points by mapping the set of structural points to the sample map; determining a deformable surface in a plane having the set of sample points, wherein each sample point of the set of sample points is represented by a surface point in the deformable surface; calculating a first deformable surface to be deformed by applying a deformation algorithm to the deformable surface, the set of structural points, and the set of surface points, wherein each surface point from the set of surface points moves to a corresponding structural point from the set of structural points; calculating a set of voxel positions relative to the first 3D image dataset or relative to a provided second 3D image dataset and based on the deformable first deformable surface; calculating a 2D unfolded image dataset of the anatomical structure based on the first 3D image dataset or the second 3D image dataset and the set of voxel positions; and providing the 2D unfolded image dataset.

Description

Technical Field

[0001] This invention relates to a computer-implemented method for providing at least one 2D unfolded image dataset, particularly of at least a portion of a hand, based on a 3D image dataset. Furthermore, the invention relates to a corresponding data processing system, a system including a medical imaging system and such a data processing system, as well as computer program products and computer-readable storage media. Background Technology

[0002] Furthermore, especially in complex accident scenarios, it is crucial to quickly understand the overall impact on the patient's body, such as to determine if and where the patient has a bone fracture or dislocation, or minor vascular injury. In these cases, three-dimensional images are typically created, for example, using computed tomography (CT) scans. In curved anatomical structures, or those that can occupy any position and posture in such scan image datasets, such as the hand, it is often difficult to assess the overall extent of damage using conventional methods. Browsing axial sections of, for example, 3D image datasets is very time-consuming, and fractures or vascular injuries often extend across multiple cross-sectional images, preventing the full three-dimensional extent of the fracture or injury from being easily visible in a single view. In contrast, three-dimensional overview images, such as volumetric rendering (VRT) techniques, visualize the surface of a structure, specifically from a particular angle. Therefore, using these methods can be cumbersome and / or prone to overlooking fractures or vascular injuries. This not only negatively impacts the patient but can also have financial and legal consequences.

[0003] One possibility for improving the visualization of such anatomical structures is to generate unfolded image datasets, which convert anatomical structures into 2D images based on 3D image datasets. For example, EP 3828836 B1 discloses a method for unfolding tubular structures, such as blood vessels. DE 102011085860 B4 discloses a method for medical imaging of body parts, particularly the hand. In this method, the image content of a curved surface is mapped onto the observation plane. Summary of the Invention

[0004] The objective of this invention is to provide an improved method comprising providing a dataset of 2D unfolded images of anatomical structures.

[0005] This task is solved by the subject matter according to the invention. Advantageous embodiments with appropriate improvements are the subject of this specification.

[0006] This invention relates to a computer-implemented method for providing at least one two-dimensional unfolded image dataset (2D unfolded image dataset) of an anatomical structure, particularly at least one partial region of the hand. In a first step, a first three-dimensional image dataset (3D image dataset) is provided, which includes the anatomical structure. In another step, a set of representative structural points of the anatomical structure is determined based on the first 3D image dataset. In another step, a 2D sample map of the anatomical structure is provided. In another step, a set of representative sample points is determined by mapping the set of structural points to the sample map. In another step, a deformable surface in a plane having the set of sample points is determined, wherein each sample point of the set of sample points is represented by a surface point in the deformable surface. In another step, a first deformable surface is deformed to determine the set of representative structural points and the set of surface points by applying a deformation algorithm to the deformable surface, wherein each surface point from the set of surface points is moved to a corresponding representative structural point from the set of representative structural points. In another step, a set of voxel positions is calculated relative to the first 3D image dataset or relative to a provided second 3D image dataset and based on the deformed first deformable surface. In another step, a 2D unfolded image dataset of the anatomical structure is calculated based on the first 3D image dataset or the second 3D image dataset and the set of voxel positions. In another step, a 2D unfolded image dataset is provided.

[0007] Here, the steps described above in the proposed method can be performed sequentially and / or at least partially simultaneously. Furthermore, the steps of the proposed method can be implemented at least partially, and especially entirely, by a computer.

[0008] Anatomical structures can be those of human and / or animal patients. However, anatomical structures can also be included in examination phantoms. In particular, an anatomical structure can include a structure whose variations in its construction within the intended patient population allow for a sample-based approach, i.e., its construction within the intended patient population shows sufficient consistency. Anatomical structures can, for example, include portions of the hand, arm, or foot. Anatomical structures can, for example, include a portion of the patient's skeletal structure. Anatomical structures can also include other anatomical structures. An anatomical structure can include multiple substructures, also referred to as subunits. If the method, for example, involves the hand, then the substructures can each include the fingers or at least all the phalanges belonging to the fingers. Particularly advantageously, the method can be used, for example, in the hand region, because there is sufficient consistency in the anatomical structure between patients, although orientation and pose in the 3D image dataset may vary particularly significantly. Mapping the image dataset of anatomical structures in an unfolded illustration can advantageously improve illustration and diagnosis.

[0009] Providing a first 3D image dataset or a second 3D image dataset may in particular include: detecting and / or reading computer-readable data storage and / or receiving from a data storage unit, such as a database. Furthermore, the image dataset may be provided by a processing unit of a medical imaging apparatus for recording the 3D image dataset. The medical imaging apparatus may, for example, include computed tomography (CT) and / or magnetic resonance imaging (MRT) and / or medical X-ray apparatus, particularly medical C-arm X-ray apparatus and / or ultrasound apparatus and / or positron emission tomography (PET) apparatus. Providing a 3D image dataset may also include generating, particularly reconstructing, a 3D image dataset based on pre-detected measurement data.

[0010] 3D image datasets allow for three-dimensional illustrations of mapped regions, particularly spatial three-dimensional illustrations. In the proposed method, the mapped region includes at least anatomical structures. 3D image datasets can also be shown as multiple layer image datasets, so-called cross-sectional views. Layer image datasets each comprise layers of the 3D image dataset at positions along marked axes. Each layer image dataset allows for two-dimensional illustrations, particularly spatial two-dimensional illustrations, of the corresponding layers of the 3D image dataset.

[0011] Advantageously, such a 3D image dataset comprises multiple voxels, in particular image points. Each voxel preferably has its own image value (and subsequently, voxel value), particularly a CT image value in HU (“Henness units”) or a similar intensity value in the case of a CT scan. Typically, the voxel centers are arranged in a regular three-dimensional grid. Similarly, a layer image dataset may comprise multiple pixels, in particular image points. Each pixel preferably has its own image value, particularly a CT image value in HU (“Henness units”) or a similar intensity value in the case of a CT scan.

[0012] The 2D unfolded image dataset has, in particular, multiple pixels, especially image points, wherein each pixel preferably has an image value, particularly a CT image value in HU (“Henness units”) or a similar intensity value in the case of a CT scan. In the proposed method, the image values ​​of the 2D unfolded image dataset are based on a pre-calculated set of voxel locations of a first or second 3D image dataset and the image values ​​of the corresponding 3D image dataset.

[0013] For example, the set of representative structural points in a first 3D image dataset can be selected (semi-)automatically and / or based on corresponding manual user input. The set of representative structural points may include one or more discrete points and / or multiple points comprising a one-dimensional line that may be curved in three dimensions. They may also form two-dimensional surfaces or variations that can be planar or curved in three dimensions. A 2D unfolded image dataset enables specific representative structural points to be shown in a single two-dimensional mapping, which, if necessary, are arranged on curved lines or surfaces in the 3D image dataset. Therefore, representative structural points can span the image plane of interest for the 2D unfolded image dataset. In particular, the set of representative structural points can map, especially in a reduced form, the three-dimensional orientation and / or orientation of anatomical structures in the 3D image dataset.

[0014] In particular, the set of representational structural points in the first 3D image dataset can be based on applying segmentation algorithms and / or edge detection and / or centerline extraction algorithms to the first 3D image dataset with respect to anatomical structures. If the anatomical structure includes, for example, at least a partial region of the hand, then the set of representational structural points may, for example, include at least one representational structural point for at least one or all fingers (especially their associated phalanges), but advantageously include multiple representational structural points, which map the orientation and / or direction of one or more fingers in the 3D image dataset. At least one, but advantageously multiple representational structural points may, for example, be located on the centerline (also referred to as the central axis) of that finger or on the centerline of the corresponding finger, especially the phalanges belonging to the finger. The same can be transferred to mapped feet or arms or other anatomical structures. Other constructions may also be provided.

[0015] Determining the set of representative structural points may also include determining additional information about the structural points, such as annotation information, the relationship between the structural points and anatomical substructures, or anatomical landmarks. If the anatomical structure includes, for example, a portion of a skeleton, then the structural points can be individually assigned to specific skeletal parts. Structural points can be "labeled."

[0016] Providing 2D sample maps (also referred to as templates or "manifestations") of anatomical structures can be detected and / or read from computer-readable data storage and / or received from data storage units, such as databases. Sample maps can, in particular, map the average anatomical structure of anatomical structures within a patient population that is affected by method averaging or is anticipated. Sample maps can therefore include standardized illustrations of anatomical structures. Sample maps can, for example, be constructed by averaging from a large sample of anatomical structures from the subjects. Sample maps can, in particular, be matched to a set of representative structural points to be determined. Sample maps can be represented in a standardized form as an image plane traversing the anatomical structure, spanned by specific representative structural points. Sample maps can, in particular, correspond to a two-dimensional illustration of the anatomical structure's anatomical structure, especially when the anatomical structure is arranged along a plane. If the method, for example, involves the hand or at least a portion of the hand, then the 2D sample map can map the anatomical structure of the hand in a standardized form, particularly the arrangement of the hand bones. Sample maps can also map the anatomical structure in a reduced form. If the representative structural points include, for example, the central axis of the anatomical structure, then the sample map can also include the sample central axis provided by the anatomical sample structure. Multiple sample maps can also be provided. For example, patient groups can be segmented into multiple categories based on patient information such as age or gender. For instance, appropriate sample maps can be selected manually or automatically based on patient data available for the current situation. In addition to a pure mapping of anatomical structures, sample maps can particularly include additional data such as annotation information, labels of anatomical substructures, or anatomical landmarks.

[0017] When determining the set of representative sample points, corresponding points of representative structural points at the same relative position within the anatomical structure can be specifically identified in the sample map. Each sample point in the set of sample points can be assigned to a representative structural point in the set of structural points. The number of sample points in the set and the number of representative structural points can correspond to a 1:1 ratio, that is, sample points can be assigned to each structural point at a 1:1 ratio.

[0018] The deformable surface may, in particular, have a smooth geometry and / or include a (surface) mesh, especially a regular mesh. The mesh may, for example, include a two-dimensional arrangement of nodes, particularly m×n nodes, where m and n are integers. The deformable surface may extend over at least the entire anatomical structure and / or a specific set of sample points. It may extend over the entire image region of a first 3D image dataset or sample image. The deformable surface may, in particular, be aligned with the sample image.

[0019] The deformable surface is specifically configured such that the set of sample points is arranged in the plane of the deformable surface, wherein each sample point is represented by a surface point in the deformable surface. The sample points of the set of sample points in the deformable surface and the surface points of the set of surface points can be the same. Therefore, surface points can be provided directly through sample points in the deformable surface. If the deformable surface is, for example, a mesh, then the surface points can correspondingly correspond to the mesh nodes of the mesh closest to the sample points. Therefore, the set of surface points can correspond to the set of mesh nodes of the sample points respectively closest to the set of sample points in the deformable surface. Other embodiments can also be provided.

[0020] Applying a deformation algorithm to an initially smooth deformable surface produces a deformable surface that is typically curved in all three dimensions, including points characterizing the structure. The deformation algorithm can, for example, be based on a so-called "as-rigid-as-Possible" algorithm. An example of this is described in Sorkine, O et al. (As-Rigid-As-Possible Surface Modeling. EUROGRAPHICS / ACM SIGGRAPH Symposium on Geometry Processing (2007), pp. 109-116). In particular, the Laplacian operator of the mesh can be preserved during deformation, resulting in the acquisition of edge lengths and angles between the mesh edges. The deformation algorithm can, for example, include nonlinear and / or iterative optimizations. Advantageously, smooth deformation of planar meshes can be ensured. Here, structure points or surface points are used as so-called "constraint points" to deform the initial undeformed deformable surface using a deformation algorithm to obtain a deformable surface including deformations characterizing the structure points.

[0021] After deformation, voxel positions can be determined, particularly voxel positions corresponding to pixels in a 2D unfolded image dataset. A subset of multiple voxels corresponding to pixels in a 2D unfolded image dataset can be determined, for example, by determining the intersection of the deformed 2D surface in three-dimensional space with multiple voxels in a three-dimensional representation of a first or provided second 3D image dataset. The subset of multiple voxels can consist of all voxels that contact or intersect with the deformed 2D surface or lie within a predefined neighborhood of the aforementioned contacting or intersecting voxels. However, other transfer methods can also be provided. Scanning can be performed by extracting voxel values ​​from voxel positions in a first or second 3D image dataset, i.e., calculating image values ​​for pixels in a 2D unfolded image, particularly a 2D unfolded image dataset. Therefore, a bidirectional assignment can be established between 3D positions in a first or second 3D image dataset and 2D positions in a 2D unfolded image dataset. Determining voxel positions or the extracted image values ​​can include interpolation, since the deformed surface does not necessarily extend through the voxel centers of the 3D image dataset, which are typically arranged in a fixed grid.

[0022] The provided second 3D image dataset can be an image dataset based on measurement data from the same recording process as the first 3D image dataset, and therefore the orientation and position of the anatomical structures in the second 3D image dataset remain unchanged relative to the first 3D image dataset. The second 3D image dataset can represent the same mapped region as the first 3D image dataset. However, the second 3D image dataset may differ from the first 3D image dataset, for example, in terms of the reconstructed image resolution or information content (e.g., involving spectral image information). If the second 3D image dataset differs from the first 3D image dataset in terms of image resolution, then, for example, for calculating voxel positions and unfolded image datasets, the side lengths of the polygons representing the planes of the deformable surface and / or the number of points scanned in each polygon can be adapted so that the resolution of the resulting 2D unfolded image dataset is adapted to the resolution of the second 3D image dataset. Advantageously, for unfolding the second 3D image dataset, the reapplication of the deformation algorithm can be abandoned, and thus the unfolded image dataset can be determined computationally and time-efficiently.

[0023] Providing a 2D unfolded image dataset may include, for example, writing the data to a computer-readable storage unit or displaying the data on a display unit, such as a monitor. It may also include providing the data via an interface to a lower-level processing unit configured for further processing of the 2D unfolded image dataset.

[0024] Providing 2D unfolded image datasets enables the acquisition of two-dimensional illustrations of complexly arranged and shaped anatomical structures from 3D image datasets. These illustrations allow for detailed examination of the spatial relationships between anatomical structures and their substructures within a single view. This illustration is particularly effective in representing a two-dimensional mapping of the anatomical structure if the anatomical structures are arranged along a plane during data acquisition, provided that the representative structural points are appropriately selected. Sample-based methods here enable a particularly efficient, robust, and simple implementation of deformation algorithms using the constrained points thus included, for deforming initial, smooth deformable surfaces. Furthermore, the simple provision and allocation of annotation information based on sample images advantageously facilitates the automated reporting of medical findings.

[0025] According to one aspect of the proposed method, determining the set of characterizing structural points may include segmentation of anatomical structures, wherein the set of structural points is determined based on the segmented structures.

[0026] Segmentation can be achieved, for example, based on pixels, voxels, edges, surfaces, and / or regions. Segmentation can also be based on model-based methods, where assumptions are made about the objects to be segmented. Segmentation can be automated or semi-automated using computational units. For example, the starting point, germ cell, or rough contour information for segmentation can be manually set. However, in preferred variants, machine learning methods, i.e., trained functions, are used for the segmentation of anatomical structures. In particular, neural networks can be used in the form of so-called U-Nets (see, for example: Ronneberger O et al. U-Net: Convolutional Networks for Biomedical Image Segmentation. https: / / doi.org / 10.48550 / arXiv.1505.04597), especially in the form of so-called nnU-Nets (see, for example: Isensee F et al. nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation. https: / / doi.org / 10.48550 / arXiv.1809.10486). However, other architectures known from existing technologies can also be used, especially those based on deep learning models with encoder-decoder architectures.

[0027] Segmentation can include segmenting the entire anatomical structure. This can include segmenting the substructures of the anatomical structure separately. If the anatomical structure includes a portion of a skeleton, then, for example, bone-by-bone segmentation may be possible. For example, each finger of the hand, and especially all the phalanges belonging to the fingers, can be segmented individually. Furthermore, segmentation of the anatomical structure can be possible in a reduced form. For example, only sub-volumes of the anatomical structure may be segmented. This could, for example, include sub-volumes around the central axis of the bone structure of the corresponding finger of the hand, for example. Especially when using a trained function, the segmentation result depends on the training data used. If the training data, as comparison data, only includes sub-volumes of the anatomical structure, then a segmentation result exists based on the trained function accordingly. The same applies to other segmentation results. Based on the segmented structure, the set of points representing the structure can be advantageously simplified and robustly, and especially automatically, determined.

[0028] Furthermore, it can be specified that segmentation is accompanied by the determination of additional information, such as annotations or labels. The segmented regions of an anatomical structure can, for example, be individually labeled as being related to substructures of the anatomical structure. Especially with the aid of trained functions, labeling, i.e., marking or annotating, the segmented regions can be done particularly simply and automatically, particularly during the segmentation step.

[0029] According to one aspect of the proposed method, the set of points characterizing the structure can represent the centerline of the anatomical structure. If the anatomical structure contains multiple substructures of interest, then multiple centerlines can also be represented. If the anatomical structure, for example, includes at least one region of the hand, then the centerline can be determined separately for each finger, and especially for the phalanges belonging to the fingers.

[0030] If the anatomical structure includes a portion of a skeletal structure, then a centerline can be determined for each bone of the anatomical structure of interest. Furthermore, providing a 2D sample map may include providing one or more sample centerlines. Determining the set of sample points is based on mapping the set of structural points onto one or more sample centerlines. Determining the set of sample points may in particular include: identifying points at the same relative positions within the sample centerlines as corresponding points representing structural points on the centerlines of the sample map.

[0031] The centerline of an anatomical structure can be advantageously determined based on the segmented anatomical structure. For example, distance transformation-based skeletonization algorithms (“DTF skeletonization”) can be applied to the segmented structure (see, for example, Boskamp T et al., Geometrical and structural analysis of vessel systems in 3D medical image datasets. https: / / www.worldscientific.com / doi / pdf / 10.1142 / 9789812701046_0001). Alternatively, the method according to Lee TC et al. (Lee TC et al., Building Skeleton Models via 3-D Medial Surface Axis Thinning Algorithms. https: / / doi.org / 10.1006 / cgip.1994.1042) or similar methods can also be used. It is also conceivable, for example, to determine the centerline based on machine learning methods with or without prior segmentation. For example, the trained function based on the nnUnet architecture mentioned above can be trained to directly output the centerline of the anatomical structure as output data. Accordingly, the trained function is trained based on training data and comparative data (including training center lines of anatomical training structures).

[0032] If annotation information, such as relevance to substructures of an anatomical structure, is detected during segmentation and / or skeletonization, it can advantageously facilitate the mapping from structural points to sample points, especially in the case of extended structures comprising multiple substructures. Therefore, substructures of an anatomical structure can be advantageously and simply assigned to substructures of the sample map.

[0033] Using a centerline in the proposed method is advantageously simple because several established algorithms can be used to extract the centerline. Furthermore, it represents a favorable and easily achievable illustration of anatomical structures in a 2D unfolded image dataset. This particularly likely means that if imaging is performed using an anatomical structure laid flat on an examination table, then the 2D unfolded image of the anatomical structure based on X-ray imaging corresponds to a coronal section passing through its center.

[0034] According to another aspect of the proposed method, additional support structure points may be determined in the first 3D image dataset, the support structure points being arranged orthogonally to the central axis in the first 3D image dataset, or if multiple central axes are determined, then arranged orthogonally to the respective central axes, and / or the support structure points are inserted into the sub-units of the anatomical structure, i.e., the transitions between sub-structures.

[0035] Additional criteria, particularly relevant to the current situation, can be set for determining the additional support structure points. Here, for example, finger orientation, finger curvature, or finger type can be considered. In the case of larger finger curvature, for example, different criteria can be set for arranging the additional support structure points compared to the case of smaller finger curvature. In the thumb, which typically exists in a different orientation than the other fingers, different criteria can be set, for example, compared to the other fingers of the hand.

[0036] Support structure points, serving as additional input points to the deformation algorithm, enable control over how the deformable surface extends through anatomical structures or their substructures. Thus, for example, it can be ensured that the deformable planar surface extends through anatomical structures, such as the corresponding bones, in the same way as the coronal plane in a standard anatomical position. Furthermore, additional support structure points in the transitions between substructures can advantageously support the normalization of how the deformable surface extends through the transition regions.

[0037] Accordingly, the support sample points in the sample image can also be identified as the corresponding points of the support structure points, and the relevant surface points in the deformable surface can be identified as additional input points for the deformation algorithm. During deformation of the deformable surface, each support surface point moves to its corresponding support structure point.

[0038] Support sample points can be assigned to corresponding sample center axes, and in particular arranged along straight lines orthogonal to the assigned sample center axes, and in the plane of the sample image. Support sample points can be assigned to each sample center axis, however, support sample points can also be assigned to only a subset. This depends particularly on the additional support structure points set in the first 3D image dataset.

[0039] According to another aspect of the proposed method, determining the set of sample points may include: aligning the sample map and the anatomical structure with each other and / or adapting the sample map to the anatomical structure.

[0040] To advantageously map characteristic structural points onto sample maps, alignment between the sample maps and the anatomical structures can be established within the 3D image dataset. 2D sample maps of the anatomical structures can be aligned, for example, using specific landmarks or oriented via central axes, such as through principal component analysis of central axes or segments. For instance, a sample map including one or more sample central axes can be placed in the 3D coordinate system of the 3D image dataset such that it is aligned with one or more central axes of the anatomical structure within the 3D image dataset, for example, by calculating least squares.

[0041] Furthermore, it can be specified, for example, that the length and / or width of the sample image or its sub-units be adapted to the anatomical structure or its substructures, in order to improve the mapping of the structural points onto the sample image. For example, the length of one or more sample central axes can be adapted to the length of one or more corresponding central axes of the anatomical structure. This can advantageously accommodate different scales compared to the initial template and avoid expansion or compression, even in the case of substructures that are physically cut or at the boundaries of the image volume.

[0042] According to another aspect of the proposed method, the method may further include: providing a third 3D image dataset including anatomical structures, locating the anatomical structures in the third 3D image dataset, determining a sub-image dataset based on the third 3D image dataset and the located anatomical structures, and providing the sub-image dataset as a first 3D image dataset.

[0043] In particular, the third 3D image dataset can map larger image segments than the first or second 3D image dataset. Specifically, the first 3D image dataset can be a sub-volume of the third 3D image dataset and is based on measurement data from the same recording process as the third 3D image dataset. For example, the third 3D image dataset can be a whole-body image dataset or an abdominal image dataset. Furthermore, especially in the case of complex accidents, large 3D image datasets are often generated to quickly understand the overall impact on the patient's body. Based on this, anatomical structures of interest can be identified or located, and sub-volumes containing these structures of interest can be used for expanded illustration and further diagnosis.

[0044] The localization of anatomical structures in a third 3D image dataset can be achieved, for example, using a trained function. This can be achieved, for example, using methods described in Zheng, Y et al. (3D Deep Learning for Efficient and Robust Landmark Detection in Volumetric Data. https: / / doi.org / 10.1007 / 978-3-319-24553-9_69), Viola P. and Jones M. (Rapid object detection using a boosted cascade of simple features. https: / / doi.org / 10.1109 / CVPR.2001.990517), or Ghesu et al. (An Artificial Agent for Anatomical Landmark Detection in Medical Images. https: / / doi.org / 10.1007 / 978-3-319-46726-9_27).

[0045] Based on the anatomical structures located in the third 3D image dataset, a partial volume of the 3D image dataset, i.e., a sub-image dataset, can be determined. This sub-image dataset specifically includes anatomical structures, where less interesting image regions from the third 3D image dataset are no longer included. Based on the sub-image dataset, more efficient and robust further processing can be advantageously performed in the proposed method.

[0046] According to another aspect of the proposed method, the first 3D image dataset may at least partially include other similar anatomical structures, wherein determining the set of points representing the structures may include separating the anatomical structures from the other similar anatomical structures.

[0047] According to another aspect of the proposed method, machine learning methods can be used in at least one step of the method.

[0048] Machine learning methods can be used, for example, in the step of determining the set of structure points and / or in the segmentation step and / or in the localization step, as described above.

[0049] Advantageously, one of these steps can be based on machine learning, particularly on applying a trained function to a first or third 3D image dataset (as input data). Here, the input data for the trained function can be based on the image dataset, particularly including image datasets. Furthermore, sets of structural points, segmented anatomical structures, or localization of anatomical structures or sub-image datasets can be provided as output data for the trained function, or can be used as input for another trained function in subsequent steps.

[0050] The trained function can be trained using machine learning methods. Specifically, the trained function can be a neural network, particularly a convolutional neural network (CNN), or a network including convolutional layers. The trained function maps input data to output data. Here, the output data can also depend on one or more parameters of the trained function. One or more parameters of the trained function can be determined and / or adapted through training. The determination and / or adaptation of one or more parameters of the trained function can be based, in particular, on paired data consisting of training input data and associated training output data, particularly comparative output data, where the trained function is used to provide the localization of a set of structural points, segmented anatomical structures, anatomical structures, or sub-image datasets. In particular, the determination and / or adaptation can be based on a comparison of training mapping data and training output data, particularly comparative output data. If the trained function is used for segmentation, for example, then a 3D training image dataset (including anatomical structures) and a comparative image dataset (including segmented anatomical structures) can be provided for training, which can then be compared with the output of the trained function. Generally, a trainable function, that is, a function with one or more parameters that have not yet been fitted, is also called a trained function.

[0051] Other terms for trained functions include trained mapping rules, mapping rules with trained parameters, functions with trained parameters, artificial intelligence-based algorithms, and machine learning algorithms. An example of a trained function is an artificial neural network, where the edge weights of the artificial neural network correspond to the parameters of the trained function. The term "neural network" is also used instead of "neural network." In particular, a trained function can also be a deep artificial neural network. Another example of a trained function is a "support vector machine," and other machine learning algorithms can also be used as trained functions.

[0052] Advantageously, the trained function, especially the neural network, has an input layer and an output layer. Here, the input layer of the trained function can be constructed to receive input data. Furthermore, the output layer can be constructed to provide mapping data, especially output data. Additionally, the input layer and / or output layer can each include multiple channels, especially neurons. Advantageously, at least one parameter of the trained function can be adapted based on a comparison dataset with a comparative dataset (e.g., a set including segmented structures or localized anatomical structures or specific structural points).

[0053] Examples of possible specific architectures that can be used in sub-steps of a method within the scope of the proposed approach have already been mentioned here.

[0054] According to another aspect of the proposed method, the method may include dividing the anatomical structure into multiple sub-units, wherein at least for each sub-unit, the steps of calculating a first deformable surface and calculating a set of voxel positions are performed individually, and wherein the 2D unfolded image dataset is calculated based on the voxel positions of the multiple sub-units.

[0055] For example, strong offsets or bends between the representative structural points of the centerline can lead to complex deformations of the deformable surface. This can result in the formation of loops or overlaps in the deformed surface, which can cause artifacts such as double representations of the structure or deformation. To avoid this, it may be advantageous to divide the anatomical structure into multiple sub-units and perform individual unfolding for each sub-unit. These sub-units are then combined to form a common 2D unfolded image dataset. For example, the fingers of a hand can each represent a sub-unit of the anatomical structure.

[0056] According to another aspect of the proposed method, at least a second deformable surface can be determined by deforming the first deformable surface along its position-dependent normal vector. A second 2D unfolded image dataset can be determined based on the deformed second deformable surface and a first or second 3D image dataset. Calculating the second 2D unfolded image dataset can be performed similarly to calculating the first 2D unfolded image dataset. In particular, two or more deformable surfaces can also be determined based on the position-dependent deformation of the first deformable surface along its position-dependent normal vector, thereby correspondingly determining two or more 2D unfolded image datasets.

[0057] By identifying at least one deformable second deformable surface and the corresponding 2D unfolded image dataset, at least one second layer of the anatomical structure can be mapped in the unfolded illustration. Advantageously, a so-called 3D stack of the anatomical structure can be provided in the unfolded illustration when calculating the deformable surfaces of multiple deformities.

[0058] If an anatomical structure is imaged using a structure laid flat on a table, then, in the case of using, for example, characterizing structural points (which represent one or more central axes of the anatomical structure), the 2D unfolded image dataset corresponds to a coronal section passing through the center. Multiple parts of the anatomical structure can be visualized from the front or back by determining a second or third deformable surface.

[0059] For example, a deformable surface mesh can be further deformed by gradually moving all mesh nodes / vertices along the direction of their normal vectors.

[0060] The unfolded image dataset can then be recreated by calculating the initial image values ​​of the first or second 3D image dataset in the same manner as described above.

[0061] Advantageously, a stack of 2D unfolded image datasets can be provided on this basis, which visualizes anatomical structures in different layers. For example, 3D illustrations (e.g., "cinematic rendering") can also be generated based on the stack of 2D unfolded image datasets.

[0062] According to another aspect of the proposed method, the provided second 3D image dataset may map the same image segments as the first 3D image dataset, but have a different image resolution and / or include spectral image information.

[0063] As previously mentioned, the provided second 3D image dataset can be an image dataset based on measurement data from the same recording process as the first 3D image dataset, and therefore, the orientation and location of anatomical structures in the second 3D image dataset remain unchanged relative to the first 3D image dataset. For example, the first 3D image dataset, or the third 3D image dataset upon which the first 3D image dataset is based, can have a coarser image resolution than the second 3D image dataset. For example, for the first or third 3D image dataset, a coarser image resolution can be reconstructed than that that can be physically and with the aid of measurement data, so that the data can be reconstructed and processed in a time-efficient manner in the proposed method. However, to compute a higher-resolution unfolded image dataset, a 3D image dataset reconstructed at a higher image resolution can still be used. Advantageously, such a high-resolution unfolded image dataset can support better visualization of small structures and potential damage.

[0064] Similarly, spectral measurement data can be used to provide 3D image datasets that include spectral information. For example, a 3D image dataset based on material decomposition can be provided. Here, a simulated 2D unfolded image dataset can also be provided, which corresponds to a two-dimensional mapping of the material decomposition map, such as an iodine map or a bone marrow map. Such image data can be displayed, for example, overlaid in a 2D or 3D visualization of the unfolded image data to highlight vascular abnormalities or microtraumatic bone injuries.

[0065] According to another aspect of the proposed method, the method may also include:

[0066] - Display the first or second 2D unfolded image dataset in the user interface.

[0067] - Determine the 2D location in the first or second 2D unfolded image dataset based on manual user input.

[0068] - Determine the 3D location in the first, second, or third 3D image dataset based on the results of 2D location and deformation algorithms.

[0069] - Display image fragments and / or cross-sectional views of the first, second, or third 3D image dataset on the user interface, including the determined 3D locations and / or markers indicating the 3D locations in the first, second, or third 3D image dataset and / or cross-sectional views.

[0070] Suggested methods may also include:

[0071] - Display a first, second, or third 3D image dataset and / or at least one cross-sectional view thereof using a user interface.

[0072] - Determine the 3D location in the first, second, or third 3D image dataset and / or cross-sectional view based on manual user instructions;

[0073] - Determine the 2D position in the first or second 2D unfolded image dataset based on the results of the 3D position and deformation algorithm.

[0074] - Display image fragments of a first or second 2D unfolded image dataset on the user interface, including the determined 2D locations and / or markers representing 2D locations in the first, second, or additional 2D unfolded image dataset.

[0075] Because advantageously there is a bidirectional allocation between the 3D positions in the first or second 3D image dataset and the 2D positions in the 2D unfolded image dataset, if a 2D position is selected in the unfolded image (“2D-3D navigation”), then the corresponding position in the initial 3D image dataset can be determined, and vice versa (“3D-2D navigation”). This can also be extended to positions in adjacent sections of the aforementioned browsable stack of multiple 2D unfolded image datasets. Advantageously, user-controlled navigation between the 3D and 2D image datasets can be implemented with corresponding illustrations via a suitable user interface (e.g., in the form of a display unit, such as a monitor or display, and a keyboard and / or mouse or including a touchscreen), which allows manual user input and display of image data.

[0076] The image segments shown can be selected, for example, centered around the location of the marker. In the case of a 3D image dataset, the image segments shown can also be presented as multiple, especially three orthogonal, cross-sectional images centered around the location of the marker. The marker may include color or other highlighting of the marker's location, for example, in the form of a symbol, and the marker is shown in the illustration, for example, overlaid with the image data.

[0077] In another aspect, the invention also relates to a providing unit configured to perform the proposed method and variations thereof.

[0078] The advantages of the proposed providing unit substantially correspond to the advantages of the proposed method for providing at least one 2D unfolded image dataset. The features, advantages, or alternative implementations mentioned herein can also be transferred to other claimed subjects, and vice versa.

[0079] The providing unit may include a computing unit, a storage unit, and / or an interface. The providing unit, particularly its components, may be configured to perform various steps of the proposed method for providing at least one first 2D unfolded image dataset and variations thereof. The interface may be configured to provide first, second, and / or third 3D image datasets. The interface may also be configured to provide 2D sample maps of anatomical structures and provide first and / or second 2D unfolded image datasets. The computing unit and / or storage unit may be used to determine a set of points characterizing the structure, determine a set of sample points characterizing the sample, determine deformable surfaces, calculate first and / or second deformable surfaces, calculate voxel positions, and calculate the 2D unfolded image dataset. Similarly, the computing unit and / or storage unit may be configured to locate the anatomical structure within the third 3D image dataset and determine sub-image datasets. The providing unit itself may also be configured to calculate, in particular, reconstruct image datasets based on the provided measurement data.

[0080] The providing unit may also include a user interface in the form of a display unit configured to show first, second, and / or third 3D image datasets and / or their cross-sectional views. This may be implemented in the form of a monitor, touchscreen, or other suitable display device. The providing unit may also include a user interface in the form of an input unit configured to enable manual user input. This may be achieved, for example, by means of a keyboard and mouse with corresponding input possibilities or by means of other suitable input units. The input unit may also be integrated into the display unit (e.g., in the form of a capacitive and / or resistive input display). Here, the input unit may also enable control of the method for providing suggestions for the 2D unfolded image dataset via user input.

[0081] In another aspect, the invention also relates to a system comprising a medical imaging apparatus and a providing unit, according to one of the above variants, for providing a 2D unfolded image dataset.

[0082] Medical imaging devices can be configured to record and / or provide first, second, and / or third 3D image datasets.

[0083] The medical imaging apparatus can preferably be configured as an X-ray imaging system, particularly a CT system, with a radiation source and a radiation detector. It can also be configured, for example, as a magnetic resonance imaging (MRT) device and / or a medical X-ray device and / or a positron emission tomography (PET) device. Furthermore, the medical imaging apparatus can be configured to record first, second, and / or third 3D image datasets of the object being examined (including regions of interest, i.e., particularly anatomical structures) and provide them to a providing unit. Alternatively, the medical imaging apparatus can be configured to record first, second, and / or third image data of the object being examined (including anatomical structures), particularly first, second, and / or third projection maps. Here, the medical imaging apparatus can also be configured to provide image data to the providing unit, wherein the providing unit is configured to reconstruct the first, second, and / or third 3D image datasets accordingly based on the first, second, or third image data and provide them to another method.

[0084] In particular, medical imaging devices can be configured as X-ray imaging systems, especially CT devices, which include photon-counting X-ray detectors. With the aid of directly converted photon-counting X-ray detectors, such as those based on semiconductor materials like CdTe or CdZnTe, small pixel sizes can be achieved in the detector, and thus high positional resolution can be realized. Advantageously, measurement data with high positional resolution can be provided, which can be transferred to a high image resolution in the reconstructed image dataset. The image resolution can here be adapted through possible prior generalizations of the measurement data and / or the reconstruction parameters used. This can advantageously facilitate improvements in diagnosis based on image datasets.

[0085] Furthermore, photon-counting X-ray detectors can be constructed to be energy-resolved. Therefore, spectral information, such as spectral information about material distribution, can also be provided. Advantageously, additional information can be provided, which can also facilitate improvements in image-based diagnostics.

[0086] The advantages of the proposed medical imaging apparatus substantially correspond to the advantages of the proposed method and / or the proposed providing unit for providing 2D unfolded image datasets. Here, the mentioned features, advantages, or alternative embodiments can also be transferred to other claimed subjects, and vice versa.

[0087] In another aspect, the invention also relates to a computer program product having a computer program that can be downloaded into the memory of a providing unit for providing at least one 2D unfolded image dataset according to one of the above variations, having program segments so that, when the program segments are executed by the providing unit, all steps of the method for providing at least one 2D unfolded image dataset and the above variations thereof are performed. The computer program product can in particular be stored on a computer-readable storage medium.

[0088] The advantages of the proposed computer program product or computer-readable storage medium substantially correspond to the advantages of the proposed method for providing a 2D unfolded image dataset. Here, the features, advantages, or alternative embodiments mentioned can also be transferred to other claimed subjects, and vice versa.

[0089] The computer program product may include, for example, software having source code or executable software code, which must be compiled and linked or simply interpreted, and the software code should be downloaded to the providing unit for execution. With the computer program product, the method for providing a 2D unfolded image dataset can be executed quickly, identically, repeatably, and robustly by means of the providing unit. The computer program product is configured such that it can perform the method steps according to the invention by means of the providing unit.

[0090] The computer program product may be stored, for example, on a computer-readable storage medium, or on a network or server, from which it can be downloaded to the processor of the providing unit, which may be directly connected to or incorporated into the providing unit. Furthermore, control information of the computer program product may be stored on an electronically readable data carrier. The control information on the electronically readable data carrier may be designed such that the control information executes the method according to the invention when used in the providing unit. Examples of electronically readable data carriers are DVDs, magnetic tapes, or USB sticks storing electronically readable control information (especially software). When the control information is read from the data carrier and stored in the providing unit, all embodiments of the method according to the invention described above can be executed.

[0091] The software-based implementation offers the following advantages: the provisioning unit, already in use, can be easily modified via software updates to operate in accordance with the invention. This computer program product may, in addition to the computer program, include, if necessary, additional components (e.g., documentation and / or additional parts) and hardware components, such as hardware keys (dongles, etc.) for using the software. Attached Figure Description

[0092] Embodiments of the present invention are shown in the accompanying drawings and will subsequently be described in detail. In different drawings, the same reference numerals are used for the same features. Wherein:

[0093] Figure 1 A schematic flowchart of a proposed method for providing at least one 2D unfolded image dataset of anatomical structures is shown.

[0094] Figure 2 A schematic flowchart illustrating the steps for determining the set of characteristic structural points according to one aspect of the proposed method is shown.

[0095] Figure 3 A schematic flowchart illustrating the steps for determining the set of characterizing sample points according to one aspect of the proposed method is shown.

[0096] Figure 4 A schematic flowchart is shown for providing a sub-image dataset as a first 3D image dataset according to one aspect of the proposed method.

[0097] Figure 5 A schematic flowchart illustrating the steps of calculating deformable surfaces based on sub-units of anatomical structures, calculating a set of voxel locations, and calculating a 2D unfolded image dataset according to one aspect of the proposed method is shown.

[0098] Figure 6 A schematic flowchart is shown for determining a second 2D unfolded image dataset based on a deformed first deformed surface according to one aspect of the proposed method.

[0099] Figure 7 A schematic flowchart illustrating one aspect of the proposed method is shown, wherein 2D positions and associated 3D positions are determined and illustrated in a 2D unfolded image dataset or a 3D image dataset.

[0100] Figure 8 Further schematic diagrams illustrating the flow of the proposed method according to implementation variations are shown.

[0101] Figure 9A schematic diagram is shown of a specific set of characterizing structural points and associated characterizing sample points in an example where the hand is used as the center line for extraction of anatomical structures.

[0102] Figure 10 A simplified schematic diagram is shown for determining a second deformation surface based on a first deformation surface according to one aspect of the proposed method.

[0103] Figure 11 A schematic diagram is shown of the method for implementing the proposal and the providing unit for its various aspects.

[0104] Figure 12 A schematic diagram of a system having a medical imaging device and a supply unit is shown. Detailed Implementation

[0105] Figure 1 A schematic flowchart is shown of a proposed computer implementation of a method for providing at least one first 2D unfolded image dataset EBDS1 of anatomical structures AS.

[0106] This method can begin with a first 3D image dataset, BDS1, which includes anatomical structures, provided in S1. The anatomical structures can be, in particular, the anatomical structures of human and / or animal patients, or included in an examination phantom. Anatomical structures may, for example, include at least one portion of a hand, arm, or foot.

[0107] In another step S2, a set of representational structural points SP of the anatomical structure is determined based on the first 3D image dataset BDS1. If the anatomical structure comprises multiple subunits, the set of representational structural points SP is particularly advantageous in representing the centerline of the anatomical structure AS or multiple centerlines of the anatomical structure AS. For example, the set of representational structural points SP may be located on the centerline of the finger, especially the phalanges belonging to the finger. The determination of the set of representational structural points SP can be performed before the segmentation S21 of the anatomical structure AS in the 3D image dataset. Particularly advantageously, the segmentation S21 can be implemented using machine learning methods, i.e., using a trained function. Particularly advantageously, the segmented anatomical structure can be used, for example, by using a skeletonization algorithm, such as DTF skeletonization, to extract one or more centerlines of the anatomical structure.

[0108] Furthermore, it is possible that the first 3D image dataset EBDS1 at least partially includes other similar anatomical structures. For example, at least one second hand may be mapped at least partially in the first 3D image dataset. In this case, the method may include: determining a set of points representing the structure, including, in particular, separating the anatomical structure from other anatomical structures after segmentation.

[0109] like Figure 2As shown, the step of determining the set of characteristic structural points SP may, for example, include: performing segmentation of anatomical structures S21 in the first 3D image dataset BDS1, performing separation of anatomical structure AS from other similar anatomical structures S22 (if any), and subsequently, for example, determining the set of characteristic structural points SP, which represents the centerline of the anatomical structure S23. In other implementation variations, step S2 may also be designed in other ways. For example, it may also be specified that the centerline is extracted directly from the first 3D image dataset BDS1, for example, using machine learning methods.

[0110] Separation S22 can be achieved here by means of the localization and segmentation of anatomical structures and the generation of masks based on the anatomical structures to mask other anatomical structures, especially after the characteristic structural points are determined. In particular, by means of trained functions, the assignment or labeling of the segmented structures as belonging to one or more anatomical structures can also be directly generated from the segmentation, on which separation can be achieved.

[0111] In a further step S3, a 2D sample diagram of the anatomical structure AS is provided. This sample diagram may also be referred to as a “template” or “model.” The sample diagram can represent a standardized illustration of the AS anatomical structure. If the method involves, for example, the hand or at least a portion of the hand, then the 2D sample diagram can map the anatomical structures of the hand, particularly the arrangement of the hand bones, in a standardized form. The sample diagram can also map the anatomical structure AS in a reduced form. If the characteristic structural points SP include, for example, the central axis of the anatomical structure, then the sample diagram can also include the sample central axis provided by the anatomical sample structure. In addition to a pure mapping of the anatomical structure, the sample diagram can particularly include additional data, such as annotation information, such as markings of anatomical substructures or anatomical landmarks. This can particularly facilitate the subsequent mapping of the characteristic structural points SP to the sample diagram.

[0112] In the next step S4, the set of representative sample points MP is determined by mapping the set of structural points SP onto the sample map. Here, corresponding points of representative structural points SP at the same relative position within the anatomical structure can be determined in particular from the sample map. If the set of representative structural points SP represents one or more center lines of the anatomical structure AS, and if providing the two-dimensional sample map in S3 includes providing one or more center lines, then determining the set of sample points MP in S4 is particularly based on the mapping of the set of representative structural points SP to the sample center lines.

[0113] According to one aspect of the proposed method, such as Figure 3As shown, determining the set of sample points MP in S4 may include, for example, aligning the sample image and the anatomical structure AS to each other in the first 3D image dataset BDS1 by means of specific landmarks or by means of central axis orientation S41. Furthermore, adapting the sample image to the anatomical structure AS S42 may include, for example, adapting the length and / or width of the sample image or its sub-units to the anatomical structure AS or its sub-structures.

[0114] In a further step S5, a deformable surface in a plane having a set of sample points MP is determined, wherein each sample point of the set of sample points MP is represented by a surface point on the deformable surface. The deformable surface may, in particular, have a smooth geometry and, in advantageous configurations, include a (surface) mesh, especially a regular mesh. If the deformable surface is set as a mesh, then the surface points may correspondingly correspond to the mesh node of the mesh that is closest to the sample point MP represented by the corresponding surface point in the initial, undeformed deformable surface. Other configurations may also be provided.

[0115] In another step S6, the first deformed surface DF1 is deformed by applying a deformation algorithm to the initial undeformed deformed surface to calculate the set of structural points SP and the set of surface points, wherein each surface point from the set of surface points moves to the corresponding structural point SP from the set of structural points SP. The deformation algorithm may, for example, be based on a so-called "as rigid as possible" algorithm, where the structural points SP or surface points are used as so-called constraint points for the deformation of the initial, undeformed deformed surface.

[0116] In additional steps S7 and S8, a 2D unfolded image dataset EBDS1 is subsequently generated based on the deformed first deformable surface and the first 3D image dataset BDS1 or the provided second 3D image dataset. In step S7, a set of voxel positions is determined relative to the first 3D image dataset BDS1 or relative to the provided second 3D image dataset and based on the deformed first deformable surface DF1. The image values ​​of the pixels in the 2D unfolded image EBDS1, particularly the 2D unfolded image dataset EBDS1, can be scanned, i.e., calculated, by extracting voxel values ​​from the voxel positions in the first or second 3D image dataset (step S8). This also establishes a bidirectional assignment between the 3D positions in the first or second 3D image dataset and the 2D positions in the 2D unfolded image dataset. Determining the voxel positions or the image values ​​extracted therefrom may include interpolation, since the deformed surface does not necessarily extend through the voxel centers of the 3D image dataset, which are typically arranged in a fixed grid.

[0117] The provided second 3D image dataset can, in particular, represent the same mapped region as the first 3D image dataset BDS1. However, the second 3D image dataset can differ from the first 3D image dataset BDS1, for example, in terms of the reconstructed image resolution or information content (e.g., involving spectral image information). The second 3D image dataset can, in particular, be based on measurement data from the same recording process as the first 3D image dataset. If the second 3D image dataset differs from the first 3D image dataset BDS1 in terms of image resolution, then for calculating voxel positions and unfolded image dataset EBDS1, it can be specified that the side lengths of the polygons representing the planes of the deformed surface and / or the number of points scanned in each polygon are adapted so that the resolution of the resulting 2D unfolded image dataset EBDS1 is adapted to the resolution of the second 3D image dataset.

[0118] In another step S9, the 2D unfolded image dataset EBDS1 is provided.

[0119] Figure 4 A variation of the method is also illustrated, in which a first 3D image dataset EBDS1 is provided based on a predetermined sub-image dataset TB from a third 3D image dataset.

[0120] This may include providing a third 3D image dataset BDS3 including anatomical structures AS in S101, locating the anatomical structures in the third 3D image dataset BDS3 in S102, determining a sub-image dataset TBS in S103 based on the third 3D image dataset BDS3 and the located anatomical structures, and providing the sub-image dataset TBS in S104 as a first 3D image dataset BDS1.

[0121] In particular, the third 3D image dataset BDS3 can map larger image segments than the first or second 3D image dataset. In particular, the first 3D image dataset BDS1 can be a subvolume of the third 3D image dataset and is based on measurement data from the same recording process as the third 3D image dataset BDS3.

[0122] For example, the localization S103 of the anatomical structure AS in the third 3D image dataset BDS3 can also be based on the application of a trained function. For example, a sub-volume, i.e., a sub-image dataset TBS including the anatomical structure, can be defined around the localized anatomical structure AS, where the sub-image dataset is provided as the first 3D image dataset BDS1 in the proposed method.

[0123] Figure 5Another method variant is shown, wherein the anatomical structure AS is divided into multiple sub-units, and for each sub-unit, at least the following steps are performed: calculating the first deformable surface DF1 of deformation S6 and calculating the set of voxel positions S7, wherein the S82D unfolded image dataset is calculated based on the voxel positions of the multiple sub-units. That is, for each sub-unit, at least the following steps are performed: calculating the first deformable surface DF1 of deformation T61, T62, T63, ... and calculating the set of voxel positions T71, T72, T73, ..., wherein, in order to calculate the S82D unfolded image dataset, partial unfolding is combined S81 to form a common unfolded image dataset.

[0124] In particular, if the anatomical structure AS includes the hand, then when divided into multiple subunits, the corresponding subunits here may include the fingers, especially all the phalanges that belong to the fingers.

[0125] Here, for partial unfolding, the support structure points can also be used as constraint points for deformation, which are assigned to adjacent sub-units to avoid inconsistent unfolding in overlapping image areas.

[0126] Figure 6 A schematic flowchart of one aspect of the proposed method is also shown, in which a second deformable surface and a second 2D unfolded image dataset are determined. For this purpose, at least one deformable second deformable surface DF2 can be determined in step S601 by deforming a first deformable surface DF1 along its position-related normal vector. Based on the second deformable surface DF2 and the first or second 3D image dataset BDS1, the second 2D unfolded image dataset can be determined. This also... Figure 10 The diagram is simplified and shown as a cross-sectional view perpendicular to the deformed surface. A proposed method, for example, generates a first deformable surface DF1, which is extracted based on the centerline of the anatomical structure AS as a characterizing structural point SP, extending through the middle layer of the anatomical structure AS. A second or third deformable surface DF2 is determined by the position-dependent deformation of the first deformable surface DF1 along its position-dependent normal vector. The layers of the anatomical structure can also be visualized from the front or back.

[0127] Based on the second deformable surface DF2, similar to the description above, a second 2D unfolded image dataset can be generated in the second 2D unfolded image dataset of the anatomical structure AS based on the second deformable surface DF2 and the calculation of the set of voxel positions of S701 relative to the first 3D image dataset BDS1 or relative to the provided second 3D image dataset, and the calculation of the set of voxel positions of the first 3D image dataset BDS1 or the second 3D image dataset and the set of voxel positions, and the second 2D unfolded image dataset can be provided in step S901.

[0128] When calculating deformable surfaces with multiple deformations in this way, the so-called 3D stack of the anatomical structure AS can be provided in particular in the unfolded illustration.

[0129] Figure 7 A schematic flowchart illustrating a method according to another aspect is shown.

[0130] Here, the method further includes: in step DS1, a first or second (and if necessary, each additional) 2D unfolded image dataset EBDS1 is displayed in user interface 41.

[0131] In addition, the method includes step DS2, which determines the 2D position in the first or second 2D unfolded image dataset EBDS1 based on manual user input.

[0132] Furthermore, in step DS3, the 3D position in the first, second, or third 3D image dataset BDS1 is determined based on the 2D position and the result of the deformation algorithm / deformed surface. In step DS4, image segments and / or cross-sectional views of the first, second, or third 3D image dataset BDS1 are displayed in the user interface 41, including the determined 3D position and / or markers representing the 3D position in the first, second, or third 3D image dataset BDS1 and / or the cross-sectional view.

[0133] Because there is an advantageous bidirectional allocation between the 3D position in the first or second 3D image dataset and the 2D position in the 2D unfolded image dataset, if a 2D position in the unfolded image is selected (“2D-3D navigation”), then the corresponding position in the initial 3D image dataset can be determined.

[0134] The same applies to the reverse path (“3D-2D navigation”). That is, the method may alternatively include: displaying a first, second, or third 3D image dataset BDS1 and / or at least one cross-sectional view thereof in step DS11 using user interface 41; determining the 3D position in the first, second, or third 3D image dataset BDS1 and / or cross-sectional view based on manual user instructions in step DS12; determining the 2D position in the first or second 2D unfolded image dataset based on the 3D position and the result of the deformation algorithm / deformed surface in step DS13; and subsequently displaying an image fragment of the first or second 2D unfolded image dataset BDS1 in user interface 41 in step DS14, which includes the determined 2D position and / or a marker indicating the 2D position in the first or second 2D unfolded image dataset BDS1.

[0135] Figure 8A flowchart of a method according to a variation of the method is shown. Here, the anatomical structure AS relates to the patient's hand. In particular, a third 3D image dataset BDS3 is provided, based on which the anatomical structure AS is located (S102) and a sub-image dataset is determined (S103), the sub-image dataset specifically including the located anatomical structure. This sub-image dataset is provided as a first 3D image dataset BDS1 (S104) for use in further steps.

[0136] Based on the first 3D image dataset BDS1, the anatomical structure AS is segmented S21, thereby providing an image volume including the segmented structure. In the illustrated case, in addition to the anatomical structure of interest, a similar second anatomical structure is further included at least partially in the image volume, thereby performing the separation of the anatomical structure S22.

[0137] Based on the separation and segmentation of anatomical structures, a set of characterizing structural points SP is determined, which represent the central lines of the segmented structures (S23), here being the central lines of the finger bones of the hand and a portion of the ulna and radius of the arm.

[0138] Furthermore, in steps S3 and S4 (not shown), sample images of the anatomical structure are provided, and a set of sample points is determined by mapping characteristic structural points onto the sample images. This may specifically include, in particular, the alignment of the sample images and the anatomical structure AS to each other S41 and / or the adaptation of the sample images to the anatomical structure AS in S42. Subsequent mapping in the method flow specifically illustrates the aligned characteristic structural points SP and associated sample points MP, which are used for subsequent unfolding. Specifically, in step S5, a deformable surface in a plane having the set of sample points MP is determined, wherein each sample point in the set of sample points MP is represented by a surface point in the deformable surface; in step S6, a first deformable surface DF1 is calculated by applying a deformation algorithm to the deformable surface, the set of structural points SP, and the set of surface points; in step S7, a set of voxel positions is calculated relative to a first 3D image dataset BDS1 and based on the first deformable surface DF1; and in step S8, a first 2D unfolded image dataset EBDS1 of the anatomical structure AS is calculated based on the first 3D image dataset BDS1 and the set of voxel positions, thereby finally providing this first 2D unfolded image dataset.

[0139] Figure 9 An aspect of the proposed method is also shown, wherein additional support structure points uSP are also determined, which are arranged orthogonally to the center lines in the first 3D image dataset, or, if multiple center lines are determined, are arranged orthogonally to the respective center lines, and / or the additional support structure points are inserted into the transition between substructures of the anatomical structure AS.

[0140] The accompanying figure illustrates extracted characterization points SP, which are arranged along the central lines of the fingers of the hand and also on portions of the ulna and radius. Furthermore, supporting characterization points uSP are determined for each central line, oriented toward and orthogonal to the extracted central lines. Additionally, supporting characterization points uSP are placed in the transition regions between the finger bones and the ulna and radius, and in the areas of the wrist bones where no central lines exist.

[0141] Additional standards can be set, especially based on the current situation, to determine additional support structure points.

[0142] The support structure point uSP is, for example, located on a line orthogonal to the extracted centerline. In the case of smaller finger curvature (e.g., <= 60º), where the finger curvature is determined, for example, by the angle between the corresponding metacarpal and the associated distal phalanx, or by the angle between vectors determined by the first and last n centerline points, the support structure point uSP is also, for example, orthogonal to the z-axis of the hand, which can be determined, for example, by PCA (principal component analysis), where the z-axis is the shortest principal axis. In the case of larger finger curvature (e.g., > 60º), the support structure point uSP is also, for example, based on a plane matching the centerline point of the corresponding finger (i.e., a plane where the sum of the squares of the distances between the point and the plane is, for example, minimized), and for the thumb, it lies within this plane, i.e., on a line orthogonal to the normal to the plane, and for the remaining fingers, it lies on a line along the direction of the plane's normal (i.e., perpendicular to the plane).

[0143] Accordingly, the supporting sample points uMP, which take the form of the sample centerline in the sample image aligned with the anatomical structure AS, are also identified as the corresponding points of the supporting structure points uSP, and the relevant surface points in the deformable surface are identified as additional input points for the deformation algorithm. The supporting sample points uMP are also oriented toward the centerline of the sample image and are arranged in the same plane as the sample image, and are located in the transition region between the finger bones and the arm bones.

[0144] The supporting structural points uSP and the corresponding supporting sample points or surface points (as additional input points for the deformation algorithm) control how the deformable surface DF1 extends through the anatomical structure AS or its substructures. Thus, for example, it can be ensured that the deformable surface DF1 extends through the anatomical structure AS, such as the corresponding bone, approximately in the same manner as the coronal plane in a standard anatomical position. Furthermore, additional supporting structural points in the transitions between substructures can advantageously support normalization, i.e., how the deformable surface extends through the transition region.

[0145] Figure 11A schematic diagram of a providing unit DV is shown, which is constructed to execute the proposed method and its various aspects. The providing unit DV may include an interface IF, a computing unit CU, and a storage unit MU.

[0146] The components providing the DV unit can be interconnected to enable efficient data exchange. The interface IF can be directly connected to the computing unit CU, which in turn can be connected to the storage unit MU. This arrangement enables information flow from the interface IF through the computing unit CU to the storage unit MU.

[0147] The DV unit can be designed for data processing and storage. The IF interface can be used here as an interface for input / output operations. The CU unit can perform computational tasks, and the MU unit can store data or results.

[0148] The interface can be specifically configured to provide first, second, or third 3D image datasets and / or sample images. The interface can be configured to provide first or second 2D unfolded image datasets. The computational unit (CU) can be configured to perform steps for determining and calculating the proposed method, and corresponding steps for implementation variations thereof.

[0149] The storage unit (MU) can be configured to store first, second, or third 3D image datasets and / or sample images and / or first or second 2D unfolded image datasets. The storage unit (MU) can use different storage technologies to enable efficient management and fast access to the stored data.

[0150] The providing unit DV may have an input unit 42 (e.g., a keyboard) and a display unit 41, such as a monitor and / or display. For example, in the case of a capacitive and / or resistive input display, the input unit 42 may be integrated into the display unit 41. Here, control of the providing unit DV can be achieved through input by a medical operator on the input unit 42. For this purpose, the input unit 42 may, for example, send signals to the providing unit DV. The display unit 41 may be advantageously configured to display 2D and 3D image datasets and information and illustrations derived therefrom, such as cross-sectional views. For this purpose, the providing unit DV may send signals to the display unit 41. Furthermore, manual input by a medical operator can be achieved by means of the input unit 42 and the possible display unit 41, for example, to determine a 2D or 3D position in a 2D or 3D image dataset according to one aspect of a proposed method, which can be further processed by means of the providing unit DV.

[0151] Figure 12 A schematic diagram of a system including a medical imaging device 32 and a providing unit DV is shown.

[0152] In this example, the medical imaging apparatus 32, particularly a CT apparatus, includes a radiation source (specifically, an X-ray source 37), a radiation detector (specifically, an X-ray detector 36), and a providing unit DV. Here, the X-ray source 37 and the X-ray detector 36 can be arranged opposite each other. The X-ray source 37 can be configured to irradiate the X-ray detector 36 with X-ray radiation along the X-ray incident direction. The X-ray detector 36 can be an indirectly converted X-ray detector, such as a scintillation detector, or a directly converted X-ray detector. A directly converted X-ray detector can, in particular, be configured as a photon-counting X-ray detector, which, for example, has CdTe, CdZnTe, CdTeSe, CdZnTeSe, or CdMnTe or other semiconductor materials as the detection material. Advantageously, the very high image resolution and / or access to spectral information achievable with a photon-counting X-ray detector can also be used within the scope of the proposed method.

[0153] The CT apparatus 32 may also include a gantry 33 with a rotor 35. An X-ray source 37 and an X-ray detector 36 may be arranged on the rotor 35 in a defined configuration, particularly integrated on or fixed to the rotor 35. The rotor 35 may be rotatably supported about a rotation axis 43. The object to be mapped (in this case, a human patient) may be positioned on a patient placement device 41 and may move along the rotation axis 43 through the gantry 32. A processing unit DV may be used to control the CT apparatus 32 and calculate cross-sectional or volumetric images of the object 39. Input devices 42, such as a keyboard and / or mouse, and output devices 41, such as a screen and / or display, may be connected to the processing unit DV, particularly coupled in signal technology. For example, in the case of resistive and / or capacitive input displays, the input device 42 may be integrated into the output device 41. The output device 41 may be configured to display a graphical display showing the image dataset and the information and illustrations derived therefrom. The input unit 42 can control the providing unit DV and / or the medical imaging device 32 through input by the medical operator on the input unit 42.

[0154] The schematic diagrams included in the accompanying drawings do not depict any scale or size ratio.

[0155] Finally, it should be reiterated that the methods and apparatus described in detail above are merely embodiments and can be modified in various ways by those skilled in the art without departing from the scope of the invention. Furthermore, the use of the indefinite articles "a" or "an" does not preclude the presence of related features multiple times. The terms "unit" and "element" do not preclude the related component from being composed of multiple interacting sub-components that may also be spatially distributed if necessary.

[0156] In the context of this application, the expression "based on" can be understood in particular in the sense of "using". In particular, the writing of generating (alternatively: obtaining, determining, etc.) the first feature based on the second feature does not preclude the generation (alternatively: obtaining, determining, etc.) of the first feature based on the third feature.

Claims

1. A computer-implemented method for providing at least one first 2D unfolded image dataset (EBDS1) of anatomical structures (AS), particularly at least one partial region of the hand, comprising the steps of: A first 3D image dataset (BDS1) is provided, which includes anatomical structures; Based on the first 3D image dataset (BDS1), determine (S2) the set of representational structural points (SPs) of the anatomical structure. Provide a 2D sample image of the anatomical structure described in (S3); The set of characterizing sample points (MP) is determined by mapping the set of structure points (SP) onto the sample graph (S4). determining (S5) a deformed surface in the plane having the set of sample points (MP), wherein Each sample point in the set of sample points (MP) is represented by a surface point in the deformed surface. The first deformable surface (DF1) is calculated (S6) by applying a deformation algorithm to the deformable surface, the set of structural points (SP), and the set of surface points, wherein each surface point from the set of surface points moves to the corresponding structural point (SP) from the set of structural points (SP). The set of voxel positions is calculated relative to the first 3D image dataset (BDS1) or relative to the provided second 3D image dataset and based on the deformed first deformed surface (S7). The first 2D unfolded image dataset (EBDS1) of the anatomical structure (AS) is computed (S8) based on the first 3D image dataset (BDS1) or the second 3D image dataset and the set of voxel locations, and Provide (S9) the 2D unfolded image dataset (EBDS1).

2. The method according to claim 1, wherein, Determining (S2) the set of characterizing structural points (SP) includes segmenting (S21) the anatomical structure (AS), and wherein the set of characterizing structural points (SP) is determined based on the segmented structure.

3. The method according to any one of the preceding claims, wherein, The set of characteristic structural points (SPs) represents the centerline of the anatomical structure (AS). Providing (S3) the 2D sample image includes providing a sample centerline, and The determination (S4) of the set of sample points (MP) is based on the mapping of the set of characterization structure points (SP) to the center line of the sample.

4. The method according to claim 3, wherein, Additional support structure points (uSPs) are also identified, which are arranged orthogonally to the centerline of the anatomical structure (AS) and / or inserted in the transition between subunits of the anatomical structure (AS).

5. The method according to any one of the preceding claims, wherein, Determining (S4) the set of sample points (MP) includes aligning the sample map and the anatomical structure (AS) to each other (S41) and / or adapting the sample map (S42) to the anatomical structure (AS).

6. The method according to any one of the preceding claims, wherein, The deformable surface is constructed as a surface mesh, and each surface point corresponds to a mesh node of the surface mesh, the mesh node being the closest to the sample point (MP) represented by the corresponding surface point in the set of sample points (MP) on the initial deformable surface.

7. The method according to any one of the preceding claims further comprises: Provide (S101) a third 3D image dataset (BDS3) including the anatomical structures (AS). The anatomical structure is located (S102) in the third 3D image dataset (BDS3); Based on the third 3D image dataset (BDS3) and the localized anatomical structure determination (S103) sub-image dataset (TBS), and The sub-image dataset (TBS) is provided (S104) as the first 3D image dataset (BDS1).

8. The method according to any one of the preceding claims, wherein, The first 3D image dataset (BDS1) includes at least a portion of other similar anatomical structures, and wherein the method comprises: Before determining the set of the characterization structural points (SP), the anatomical structure (AS) is separated (S22) from the other similar anatomical structures.

9. The method according to any one of the preceding claims, wherein, Machine learning methods should be used at least in the steps of the method described.

10. The method according to any one of the preceding claims, wherein, The anatomical structure is divided into multiple sub-units, and for each sub-unit, at least the following steps are performed: calculating (S6) the deformed first deformable surface (DF1) and calculating (S7) the set of voxel positions, wherein the 2D unfolded image dataset is calculated (S8) based on the voxel positions of the multiple sub-units.

11. The method according to any one of the preceding claims, wherein By deforming the first deformed surface (DF1) along its position-related normal vector, at least one deformed second deformed surface (DF2) is determined (S601), and Based on the second deformable surface (DF2) and the first 3D image dataset or the second 3D image dataset (BDS1), determine (S701, S801) the second 2D unfolded image dataset.

12. The method according to any one of the preceding claims, wherein, The provided second 3D image dataset maps the same mapping region as the first 3D image dataset (BDS1), but has a different image resolution and / or includes spectral image information.

13. The method according to any one of the preceding claims, wherein The first 2D unfolded image dataset or the second 2D unfolded image dataset (EBDS1) is displayed in the user interface (41) (DS1). The 2D position in the first 2D unfolded image dataset or the second 2D unfolded image dataset (EBDS1) is determined based on manual user input (DS2). Based on the 2D position and the result of the deformation algorithm, determine the 3D position in the first 3D image dataset, the second 3D image dataset, or the third 3D image dataset (BDS1) (DS3). The user interface (41) displays image fragments and / or cross-sectional views of the first 3D image dataset, the second 3D image dataset, or the third 3D image dataset (BDS1), the cross-sectional views including determined 3D locations and / or markers representing the 3D locations in the first 3D image dataset, the second 3D image dataset, or the third 3D image dataset (BDS1) and / or the cross-sectional views. Or among them, The first 3D image dataset, the second 3D image dataset, or the third 3D image dataset (BDS1) and / or at least one cross-sectional view thereof are shown via a user interface (41) (DS11). Based on manual user instructions, determine the 3D positions in the first 3D image dataset, the second 3D image dataset, or the third 3D image dataset (BDS1) and / or the cross-sectional view. Based on the 3D position and the result of the deformation algorithm, determine the 2D position in the first 2D unfolded image dataset or the second 2D unfolded image dataset (DS13), and The user interface (41) displays an image fragment of the first 2D unfolded image dataset or the second 2D unfolded image dataset (EBDS1), the image fragment including the determined 2D position and / or a marker representing the 2D position in the first 2D unfolded image dataset or the second 2D unfolded image dataset (EBDS1).

14. A providing unit (DV) configured to perform the method according to any one of the preceding claims.

15. A system comprising: Medical imaging devices (32), preferably X-ray imaging devices, especially CT devices, have a radiation source (37) and a radiation detector (36), and The providing unit (DV) for providing a 2D unfolded image dataset (EBDS1) as described in claim 14.

16. A computer program product having a computer program capable of being downloaded to a memory (MU) of a providing unit (DV) for providing at least one first 2D unfolded image dataset (EBDS1) according to claim 14, having program segments to perform all steps of the method according to any one of claims 1 to 13 when the program segments are executed by the providing unit (DV).

Images