Anatomical axis, landmark or plane prediction based on a three-dimensional model of an anatomical structure

Abstract

The present invention relates to a method for predicting at least one axis, plane or point from medical data representing one or more anatomical structures. The method comprises: obtaining (11, 12) one or more medical 3D dataset volumes representing the medical data and comprising one or more complete or incomplete anatomical structures; 5 segmenting (13) the one or more 3D dataset volumes to obtain a digital 3D representation of the anatomical structure; and predicting (15, 16, 17) by using a pre-trained machine learning network at least one axis, plane and / or point from the digital 3D representation as processed or unprocessed. The axis, plane and / or point represent(s) information at least not fully contained in the digital 3D representation. The network is 0 pre-trained on training data comprising complete or incomplete models of the anatomical structure with axis, plane or point information gained during training from a complete model of the anatomical structure.

Description

[0001] ANATOMICAL AXIS, LANDMARK OR PLANE PREDICTION BASED ON A THREE- DIMENSIONAL MODEL OF AN ANATOMICAL STRUCTURE

[0002] FIELD OF THE INVENTION

[0003] The present invention relates to a method for predicting an axis, landmark, and / or plane of an anatomical structure from an incomplete digital 3D representation of the anatomical structure, which may be for instance a bone, such as a human scapula. The present invention also relates to an apparatus for implementing the proposed method.

[0004] BACKGROUND OF THE INVENTION

[0005] The analysis of anatomical structures is indispensable in a multitude of medical fields, encompassing diagnostic procedures, prognosis determination, pathology prevention strategies, and the formulation of treatment decisions, as well as the planning and optimisation of therapeutic interventions. For example, in the realm of patientspecific shoulder prosthesis planning, it is important to accurately parameterise the shape of the scapula. This detailed analysis includes the calculation of axes, landmarks and planes describing the scapular morphology, to ensure that a prosthesis will fit optimally and function effectively. Moreover, assessing the orientation of the elbow relative to the humeral head orientation is a vital aspect of comprehensive shoulder joint evaluation, which aids in achieving proper joint alignment and function.

[0006] Morphological analysis can be performed on medical images of the structure. Clinical imaging data, however, often comes with inherent limitations. The field of view may be restricted for example, due to the need to balance acquisition time with image resolution, leading to partial visualisation of the anatomical structure. In such cases, essential structures that lie outside the limited field of view are not captured in the imaging process, resulting in incomplete datasets. These restrictions can impede the precise definition of critical axes and planes required for morphological analysis. As an example, three-dimensional (3D) morphology assessment approaches, in which anatomical landmarks are depicted on a 3D surface model of the scapula / humerus / pelvis / knee / elbow generated from semantic segmentation of the structures from computed tomography (CT), have been proposed. However, the associated cost and irradiation of additional CT acquisition prevents such techniques from being used in clinical routines for all patients. For effective 3D measurement on magnetic resonance imaging (MRI), accurate segmentation of the anatomical structure to create a 3D surface model, as well as the depiction of bony landmarks is often required. However, MRI analyses are often limited compared with CT by their restricted field of view, often failing to include an entire critical structure, such as the complete scapular body for shoulder analysis. In the case of diagnostic shoulder MRI with restricted field of view, for example, the medial and inferior scapular boarders, which are necessary to define axes, landmarks, and planes used to describe the scapular morphology for implant position planning, are often cropped. Therefore, traditional morphology calculations based on automatic landmark detection would not be applicable.

[0007] Other examples include the need for calculation of the orientation of the elbow with respect to the orientation of the humeral head, required for shoulder arthroplasty implant positioning planning or the calculation of the orientation of the knee with respect to the femoral head for the planning of hip procedures. In such cases, additional images of the elbow and knee are required just to evaluate these orientations because image data from the shoulder and hip are restricted.

[0008] Furthermore, an analysis of the complete anatomical structure may also not be possible even if it is captured within the CT or MRI if, for example, the image quality in part of the image is insufficient for complete structure segmentation or definition. For example, the wing of the scapula is often too thin to be defined / segmented in MRI (if for example the MRI resolution is less or similar to the wing thickness).

[0009] To date no method exists to predict the orientation of the elbow with respect to the humeral head only from image data of the shoulder joint without the need of additional imaging of the elbow, nor to predict the relative orientation of the knee to the femoral head without additional imaging for the knee. More broadly, to date no method for describing the full anatomical structure morphology in the case of incomplete structure imaging has been proposed.

[0010] BRIEF DESCRIPTION OF THE INVENTION

[0011] The present invention aims to overcome at least some of the above-identified problems related to anatomical axis, landmark, and / or plane determination of an anatomical structure based on incomplete medical image data with a restricted field of view for instance. The proposed computer-implemented method and system may be used to accurately define anatomical reference axes, landmarks, and / or planes despite incomplete definitions of anatomical structures, thus providing enhanced diagnostic and / or prognostic capability and seamless integration into the clinical decision-making workflow. According to a first aspect of the invention, there is provided a method for predicting at least one axis, point or plane from medical data representing one or more anatomical structures as recited in claim 1.

[0012] The proposed method addresses and resolves the existing drawbacks in anatomical structure analysis on medical data with incomplete image data, which may include image data with a restricted field of view. By integrating advanced artificial intelligence (Al) techniques, the proposed approach can accurately predict axes, landmarks, and / or planes of entire anatomical structures even from incomplete imaging data. This reduces the need for additional imaging, thereby cutting costs and minimising patient exposure to radiation. These predictive models overcome the limitations of restricted field of view in MRI and CT scans. As a non-limiting example, the proposed method enables the prediction of anatomical orientations, such as the orientation of an elbow relative to the humeral head, from image data solely covering the shoulder joint but not visualising the elbow, eliminating the need for additional scans of the elbow. These innovations may collectively provide more accurate, efficient, and holistic assessments, improving subsequent diagnostic procedures, treatment planning, and therapeutic interventions.

[0013] According to a second aspect of the present invention, there is provided a computer program product comprising instructions for implementing the steps of the method when loaded and run on a computing apparatus.

[0014] According to a third aspect of the invention, a data processing system is provided, which comprises means for carrying out the method according to the first aspect of the present invention.

[0015] Other aspects of the invention are recited in the dependent claims attached hereto.

[0016] BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The invention will now be described in more detail with reference to the attached drawings, in which:

[0018] • Figure 1 shows a block diagram schematically illustrating an example data processing system where the teachings of the present invention may be applied;

[0019] Figure 2 shows a flowchart illustrating the proposed method according to an example embodiment of the present invention; • Figure 3 shows an anatomical structure in a restricted field of view;

[0020] • Figure 4 illustrates how different 3D surface models taken along different orientations can be combined into a single high-resolution 3D surface model;

[0021] • Figure 5 illustrates the step of increasing a binary mask volume in the proposed method;

[0022] • Figure 6 shows 3D surface models of two mutually different anatomical structures, where the left-hand side of the figure shows complete 3D surface models with axes defined by two points on the surface of the models, whereas the right-hand side of the figure shows incomplete 3D surface models of the same anatomical structures as the left-hand side but with axes representing information not contained in the incomplete 3D surface models; and

[0023] • Figure 7 illustrates the training process of a machine learning network of the system shown in Figure 1 , and the application of the machine learning network to subsequent new data.

[0024] DETAILED DESCRIPTION OF THE INVENTION

[0025] An embodiment of the present invention will now be described in detail with reference to the attached figures. It should be noted that the figures are provided merely as an aid to understanding the principles underlying the invention and should not be taken as limiting the scope of protection sought. Where the same reference numbers are used in different figures, these are intended to indicate similar or corresponding features. As utilised herein, “and / or” means any one or more of the items in the list joined by “and / or”. As an example, “x and / or y” means any element of the three-element set {(x), (y), (x, y)}. In other words, “x and / or y” means “one or both of x and y.” As another example, “x, y, and / or z” means any element of the seven-element set {(x), (y), (z), (x, y), (x, z), (y, z), (x, y, z)}. In other words, “x, y and / or z” means “one or more of x, y, and z.” Furthermore, the term “comprise” is used herein as an open-ended term. This means that the object encompasses all the elements listed, but may also include additional, unnamed elements. Thus, the word “comprise” is interpreted by the broader meaning “include”, “contain” or “comprehend”. As used herein, unless otherwise specified, the use of the ordinal adjectives ’’first”, ’’second”, “third”, etc. to describe a common object, merely indicate that different instances of like or different objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

[0026] Figure 1 schematically illustrates an example imaging or data processing system 1 , which is configured to carry out the proposed method as described later in more detail. The system 1 comprises a data processing unit 2, which can access a memory unit 3 configured to store a set of medical images of an anatomical region of a subject. The system also comprises a machine learning network or a piece of Al software 4 (i.e., an Al application), which may for instance be an artificial neural network, such as a convolutional neural network. The system in this case further comprises a user interface 5 to receive user inputs from an operator, and a data output unit 6, which may comprise a display and / or means to output digital information, such as an input / output port.

[0027] A non-limiting example embodiment of the proposed method is next explained in more detail with reference to Figure 2. At step 11 , raw image data, which may be a series or set of images (including single images), are acquired or accessed. The raw image may be acquired using a medical imaging system, such as an MRI or CT system, or the data may be accessed from a database, such as a medical image archive. The images in the raw image dataset may be 2D and / or 3D images. Data pre-processing is performed at step 12 to generate a corrected 3D volume. Images taken during clinical workflow may suffer from large variabilities in imaging protocols, especially the resolution and field of view. For an example, in a sagittal MRI dataset of the shoulder, the most medial image slice may end before the lateral joining of the scapular spine with the scapular structure, while other MRI datasets may exhibit an almost complete coverage of the scapula. These variations may be addressed at pre-processing step 12, containing image interpolation and resizing the entire image to fit a single predetermined size.

[0028] At step 13, the corrected 3D volume containing one or more anatomical structures, which in this example are one or more bones, is segmented to obtain a segmented or labelled volume of the anatomical structure, i.e., the segmented or labelled volume shows the anatomical structure. Volume segmentation is a process used to partition or label a volume into multiple regions or segments (in this case into two segments so that a first segment is identified with a first bit value, while a second, different segment is identified with a second, different bit value), making it easier to analyse and interpret specific structures within the volume. More specifically, at step 13, the 3D volume is segmented by labelling the one or more anatomical structures with a first label, and remaining elements (i.e. , the background) in the 3D volume with at least a second, different label to obtain a digital 3D representation of the one or more anatomical structures. In this example, the first label has a bit value of 1 , and the second label has a bit value of 0, or vice versa. Thus, in the present example, the obtained digital 3D representation is a binary mask volume, which is a 3D volume highlighting the anatomical structure with a given bit value. However, it would be possible to have more than one anatomical structure inside a mask volume, where a respective anatomical structure is represented with its individual integer value. In this example, the first label has a first voxel value, the second label has a second, different voxel value, and so on.

[0029] The following segmentation types are available:

[0030] • Manual segmentation involves a radiologist or technician manually outlining regions of interest. While accurate, it is time-consuming and subject to inter- and intra-observer variability.

[0031] • Semi-automatic segmentation combines manual input with automated processes to improve efficiency and consistency.

[0032] • Automatic segmentation: uses algorithms to automatically identify and segment regions. This method aims to be both time-efficient and reproducible.

[0033] In the present example, the segmentation is carried out by the network 4, which in this example is a deep convolutional neural network (DCNN). Alternatively, the segmentation of the specific anatomical structure could also be done manually or semi- automatically. Manual correction of automatic segmentation could also be performed. In some configurations, to validate performance of the automatic segmentation, comparison models may be used. A comparison model may include a network which performs multiple segmentations of different anatomical bodies at once. Another comparison model may be an Al network trained on different training data.

[0034] Referring to Figure 3, as a non-limiting example, the binary mask volume 20 of the anatomical structure 21 could be incomplete because the field of view of the medical image data does not include the whole structure. A 2D representation is illustrated in Figure 3 for simplicity. The left-hand side of Figure 2 shows the complete anatomical structure 21 , while the right-hand side of Figure 3 shows the incomplete anatomical structure 21. Another reason for an incomplete anatomical structure could be that a segmentation at a certain position is not feasible due to the resolution of the image and the structure of the anatomical structure. Post-processing may be performed on the binary mask volume 20.

[0035] If a plurality of anatomical structures are examined, a plurality of binary mask volumes could be used where each anatomical structure would be stored or shown in an individual binary mask volume or multiple structures could be stored in one single mask volume, where each voxel is assigned either to the background or one of the anatomical structures with different (integer) values.

[0036] If highly non-isotropic (having mutually different resolutions in different directions) or low-resolution data of the anatomical structure 21 is available, including but not limited to coronal, transversal, or sagittal MRI scans, additional steps may optionally be applied to increase the accuracy of the proposed method. To create a high-resolution representation of the anatomical structure 21 , the information of multiple binary mask volumes 20 of the same anatomical structure from multiple acquisitions in different orientations may be combined. Thus, steps 11 to 13 may be carried out for multiple, differently oriented image data of the same structure to generate individual binary mask volumes 20 along different orientations to be combined to obtain the final, combined binary mask volume 20. This is illustrated in Figure 4 showing three 3D surface models of three binary mask volumes of a scapula obtained from an MRI scan acquired in different orientations or planes (coronal, sagittal, and axial) as well as a 3D surface model of an isotropic high-resolution binary mask volume of the scapula after combining the information from the different orientations. The methodology for such information integration may include advanced interpolation techniques, deep learning based or traditional multi-planar reconstruction algorithms, and other image processing methods designed to enhance the spatial resolution and fidelity of the final high-resolution binary mask volume used to generate the corresponding surface model. For accurate combination of the information of the binary volumes along the different orientations, methods to consolidate the regions of highest accuracy from each of the low-resolution, anisotropic binary volumes may be applied. Specifically, these regions could be identified voxel-wise by consistency maps of the binary mask border across adjacent slices in the direction with low resolution. These consistency maps could be used as voxel-wise weighting during the combination process of the different orientations. Alternatively, convolutional neural network algorithms with MRIs along different orientations as inputs may be used to combine the information and create isotropic high-resolution binary volumes. However, it is to be noted that the proposed method may instead be used directly on a low-resolution representation of the anatomical structure 21. The following steps of the method allow at least one axis 22, landmark 23, or plane 24 to be defined. More specifically, these steps allow for the determination of at least one axis, landmark, or plane, the accurate determination of which requires information outside the binary mask volume 20 containing the anatomical structure(s) 21 . If the original binary mask volume does not contain one or more voxels at the location where the axis, landmark or plane will be defined, the volume may optionally be increased at step 14 towards the direction of the missing part of the structure using voxel padding with the value of the background or with other values (excluding in this case the label value used to identify the anatomical structure) to allow for a prediction outside of the original size of the anatomical structure. The new increased volume size may be based on an estimate of the size of the complete anatomical structure under examination. Figure 5 visualises in a non-limiting example the padding of the binary mask volume 20 to allow for a prediction outside of the original size of the anatomical structure 21.

[0037] The prediction of at least one axis 22, landmark 23, and / or plane 24 of the anatomical structure 21 using the network 4 is performed next at step 15. This evaluation may be based on a single complete or incomplete anatomical structure, or multiple complete or incomplete anatomical structures. The network 4 is here used to segment 3D volumes containing geometrical objects to thereby show up at least one axis 22, landmark 23 or plane 24. For example, geometrical binary masks of spheres could be used to represent landmarks, rods could be used for axes, and discs or plates could be used for planes. Thus, in the present example, the pre-trained neural network 4 is used for automatic generation of geometrical binary masks of at least one geometrical object. In this example, the geometrical binary masks, which are connected voxel representations or formations of geometrical structures of spheres, rods and / or discs, form the output of the network 4. In other words, at step 15, the binary mask volume, as increased or non-increased in size, is input to the network 4 to predict binary labels representing a sphere, a rod, and / or a disc or plate within a 3D volume of the same size and resolution as the input binary mask volume 20. The binary sphere mask for landmark definition may have a diameter between single voxels and several millimetres. The binary rod mask for axis definition may have a diameter between single voxels and several millimetres. The binary disc or plate mask for plane definition may have a thickness between a single voxel and several millimetres. The axes and planes may be partially overlapping with the complete or incomplete anatomical structure(s), i.e., a part of the respective axis or plane may extend into the anatomical structure(s). In the cases where multiple axes, landmarks and / or planes are predicted, a single network to predict all geometries may be trained. Alternatively, multiple networks may be trained, each network only predicting one or more geometries which afterwards may be combined.

[0038] Post-processing is optionally carried out at step 16. Post-processing may be required to correct any fragmented or extraneous predictions and may thereby improve the overall accuracy and reliability of the predictions of the geometrical structures obtained at step 15. As a non-limiting example, the post-processing workflow may involve reducing false positives by removing all but the largest component of the predicted sphere, rod, and / or plane masks, where the largest component refers to the largest formation or combination of connected voxels. Additionally, connection analysis and binary closing may be applied to ensure that each geometrical structure is represented as a connected, dense, and closed 3D volume.

[0039] At step 17, the point position, axis origin and orientation, and / or plane origin and orientation are extracted or defined from the respective geometrical binary mask volume of the geometrical structure. For the points, this may be done by extracting the centre of the binary mask of the sphere. To extract the axis and plane origins and orientations a principal component analysis of the geometrical binary masks of the rod, discs or plates may be carried out. If required, the origin of the axis and / or planes may be adjusted to lay on the surface of the anatomical structure by moving the initial origin along the axis or plane, respectively.

[0040] The above-described method may be used to define or determine at least one axis 22, landmark 23 or plane 24, where the axis, landmark or plane represents information not contained or not fully contained in the digital representation of the anatomical structure segmented at step 13. For example, the axis, landmark, or plane may represent a part of the anatomical structure segmented at step 13 but not contained in the segmentation or the medical data, i.e., the raw image data. The axis, landmark, or plane may also or instead represent another anatomical structure than the one segmented at step 13. The axis, landmark, or plane may also or instead represent a general landmark, axis, or plane not directly related to any anatomical structure defined by the medical data. As non-limiting examples, Figures 3 and 6 show two landmarks 23, namely a first point P1 and a second point P2, on an anatomical structure 21 , and which are required to define a specific axis 22. While P1 lays inside the field of view and could be defined based on the digital representation of the anatomical structure alone, P2 lays outside this field of view and is not represented in the medical data or the digital representation of the anatomical structure.

[0041] In some configurations, to validate the performance of steps 15 to 17, comparative methods may be used. The results of different methods may be compared, and the degree of agreement may be used as an indication of their performance. A comparison model may be one or more networks trained to define other sets of axes, landmarks, and / or planes or one or more networks trained on different training data. In other words, the axis, landmark and / or plane definition(s) may be carried out by using at least a first network configuration, and a second, different network configuration. If the results obtained from these network configurations are very similar, then we can be confident that the outcome of the method is accurate.

[0042] To train the network 4, a training dataset may be used which contains the required information for accurate definition of at least one axis, landmark, or plane. This information may be available due to external, additional measurements or from data with a field of view containing the part of the anatomical structure required for the definition of at least one axis, landmark, or plane. The network 4 may be trained on binary mask volumes of complete or incomplete anatomical structures that share the same field of view as the data on which the network will be applied after training (referred to hereinafter as new data, and sometimes referred to as validation data or test data). If training data has a larger field of view than the field of view of new data, voxel values in the binary mask volume outside of the field of view of the new data may be set to the background value in the training data. During training, data augmentation methods may be used to improve the performance and generalisation of the network by randomly rotating, flipping, translating, zooming and / or shrinking the image content during training.

[0043] For example, the Al algorithm or model of the network 4 may be trained on data where the full scapula is visualised in the image, and a complete 3D model and binary mask volume of the scapula can be generated allowing for accurate axis, landmark, or plane definition. The network may be trained on complete or cropped scapula volumes with the axis, landmark, or plane gained from the complete model, for example by using computing means which are not part of the network 4, and then used to train the network.

[0044] As a non-limiting example, Figure 7 illustrates the training process of the model and its subsequent application to new data. During training, the larger field of view of the training data can be first used to define an axis, landmark and / or plane of the anatomical structure before feeding any training data to the network 4. The training data represents the same anatomical structure but typically from different individuals as the new data used to obtain a prediction. To train the network, voxels in the binary mask volume outside the field of view of new data are set to zero (or to a value of the background). The network 4 is then trained to predict the axis, landmark and / or plane within the volume from the binary mask volume of the clipped structure. For new data with a restricted field of view, zero-padding is applied to match the size of the training data. The trained network 4 is then used to predict the axis 22, landmark 23, and / or plane 24 from the binary mask volume 20 of the smaller field of view.

[0045] In some configurations, at least one axis, landmark, and / or plane may be used for the calculation of angles and / or distances between the segmented anatomical structure and one or more surrounding structures not comprised in the medical data. In this case, it would be advantageous if these different structures do not move relatively to each other. It would also be possible to calculate an angle between two positions of the anatomical structure 21. In some configurations, at least one axis, landmark, and / or plane overlayed on the 3D surface model of the segmented anatomical structure may be visualised. In some configurations, morphological parameters of an anatomical structure using at least one axis, landmark, and / or plane may be calculated. For the scapula, morphological parameters are for instance the glenoid inclination and version, the critical shoulder angle or novel 3D morphological metrics. The humeral retroversion and torsion considering the orientation of the humeral head and the elbow could also be calculated. For the hip, the mechanical lateral distal femoral angle defined from the femur mechanical axis (axis connecting the centre of the femoral head and the centre of the distal femoral epiphysis) and the knee joint line of the femur could be calculated.

[0046] In some configurations, comparative methods may be used to validate performance of the automatic definition of axes, landmarks and / or planes. The results of different methods may be compared, and the degree of agreement may be used as an indication of their performance. As a non-limiting example, a method to define at least one axis 22, landmark 23, and / or plane 24 from one or more complete or incomplete anatomical structures may be implemented by the network 4 or the like to directly predict the 3D position of the landmark, and / or the origin and orientation of the axis and / or plane, which in this example are the outputs of the network 4. In this case, the network 4 could be trained to directly define the axis, landmark, and / or plane from the binary mask volume or the 3D surface model of the anatomical structure. The architecture of this network could be considerably different from the one used at step 15. It is to be noted that a 3D surface model of an anatomical structure can be created from a (binary) mask volume and the voxel spacing using the marching cube algorithm.

[0047] The processes described herein may be executed by a computing system, such as the one shown in Figure 1 , or by another processor. Artificial intelligence (Al) techniques can be integrated into the software within the network 4 and accessed by the processor 2 to predict at least one axis, landmark, and / or plane. Different network architectures may be employed. For example, an Al model of the network 4 could utilise a 3D U-Net structure according to the teachings of publication entitled “3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation ” by Ozgun igek et al. designed for segmentation, incorporating an encoder-decoder framework with multiple resolution levels for 3D images. The encoder may include convolutional, batch normalisation, and activation layers, followed by pooling layers. The decoder may include deconvolution layers and similar convolutional blocks, with skip connections linking encoder and decoder layers. The final layer adjusts feature dimensions to match the number of anatomical structures to be segmented, followed by a voxel-wise classifier.

[0048] Implementation may leverage a neural network framework, such as TensorFlow. Training and testing may be conducted on graphics processing units. During training, weights may be initialised randomly from a Gaussian distribution and updated using an adaptive moment estimation (Adam) optimiser for gradient descent, with a predefined initial learning rate. Extensive data augmentation may be applied during training, such as random rotation, flipping, resizing, shearing to allow for better generalisation of the network. The loss function may include voxel-wise dice loss and cross-entropy.

[0049] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive, the invention being not limited to the disclosed embodiment. Other embodiments and variants are understood and can be achieved by those skilled in the art when carrying out the claimed invention, based on a study of the drawings, the disclosure and the appended claims. New embodiments may be obtained by combining any of the teachings above.

[0050] In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used. Any reference signs in the claims should not be construed as limiting the scope of the invention.

Claims

CLAIMS1 . A method for predicting at least one axis (22), plane (24) or point (23) from medical data representing one or more anatomical structures (21 ), the method comprising:• obtaining (11 , 12) one or more 3D dataset volumes representing the medical data and comprising one or more complete or incomplete anatomical structures (21);• segmenting (13) the one or more 3D dataset volumes by labelling the one or more anatomical structures (21) with a first label, and remaining elements in the one or more 3D volumes with at least a second, different label to directly or indirectly obtain a digital 3D representation (20) of the one or more anatomical structures (21 ); and• predicting (15, 16, 17) by using a pre-trained machine learning network (4) at least one axis (22), plane (24) and / or point (23) from the digital 3D representation (20) as processed or unprocessed, wherein the at least one axis (22), plane (24) and / or point (23) represent(s) information at least not fully contained in the digital 3D representation (20), wherein the machine learning network (4) is pre-trained on training data comprising complete or incomplete models of the one or more anatomical structures (21), and axis, plane or point information is gained during training from one or more complete models of the one or more anatomical structures (21 ) and used to train the machine learning network (4).

2. The method according to claim 1 , wherein the one or more 3D dataset volumes comprise 3D dataset volumes taken along mutually different orientations, and wherein to increase the resolution of the digital 3D representation the method further comprises combining the segmented 3D dataset volumes taken along mutually different orientations to obtain the digital 3D representation (20).

3. The method according to any one of the preceding claims, wherein the machine neural network (4) is an artificial neural network (4), and / or wherein the medical data comprises computed tomography medical data and / or magnetic resonance imaging medical data.

4. The method according to any one of the preceding claims, wherein the machine learning network (4) is trained so that if the training data have a larger field of view than the field of view used to obtain the medical data, voxel values in the one or more 3Ddataset volumes of the training data outside of the field of view used to obtain the medical data are set to a label value identifying the remaining elements.

5. The method according to any one of the preceding claims, wherein the digital 3D representation (20) is a binary representation, and wherein the first label is a first bit value, and the second label is a second, different bit value.

6. The method according to any one of the preceding claims, wherein the digital 3D representation (20) is a 3D mask volume or a 3D surface model.

7. The method according to any one of the preceding claims, wherein if the one or more 3D dataset volumes do not contain one or more voxels at the location where the axis, plane, or point of the anatomical structure(s) are to be predicted, the method further comprises increasing (14) the digital 3D representation (20) or the one or more 3D dataset volumes towards the direction of the missing part(s) of the anatomical structure(s) using voxel padding with a label value identifying the remaining elements and / or with at least another label value.

8. The method according to claim 7, wherein the new increased volume size is based on an estimate of the size of the complete anatomical structure(s) under examination.

9. The method according to any one of the preceding claims, wherein the step of predicting by using the pre-trained machine learning network (4) at least one axis (22), plane (24) and / or point (23) from the digital 3D representation as processed or unprocessed comprises segmenting (15) the digital 3D representation as processed or unprocessed to obtain one or more geometrical mask volumes of at least one rod to represent the axis (22), at least one disc or plate to represent the plane (24), and / or at least one sphere to represent the point, and wherein the respective geometrical mask volume comprises a set of connected voxels, and wherein the method further comprises extracting (17) at least a point position, an axis origin and / or orientation, and / or a plane origin and orientation from the one or more geometrical mask volumes.

10. The method according to claim 9, wherein after having obtained the geometrical mask volumes, the method further comprises post-processing (16) the one or more geometrical mask volumes by reducing false positives by removing all but thelargest connected voxel component for the sphere, rod, and / or disc or plate, and / or by performing a connection analysis of voxels of the one or more geometrical mask volumes to ensure that each geometrical structure in the one or more geometrical mask volumes is represented as a connected, dense, and closed 3D volume.

11. The method according to claim 9 or 10, wherein the extraction is carried out by a processing unit (2) operatively connected to the machine learning network (4).

12. The method according to any one of claims 1 to 8, wherein the step of predicting by using the pre-trained machine learning network (4) at least one axis (22), plane (24) and / or point (23) from the digital 3D representation as processed or unprocessed comprises directly defining at least a point position, an axis origin and / or orientation, and / or a plane origin and orientation from the digital 3D representation as processed or unprocessed.

13. The method according to any one of the preceding claims, wherein the method further comprises at least one of the following:• calculating morphological parameters from the predicted axis (22), plane (24) and / or point (23);• overlying the predicted axis (22), plane (24) and / or point (23) on a 3D surface model derived from the digital 3D representation as processed or unprocessed, and visualising the predicted axis (22), plane (24) and / or point (23) on the 3D surface model;• calculating an angle and / or a distance between the one or more anatomical structures (21 ) and one or more other anatomical structures not represented by the medical data, and / or calculating an angle between two positions of the one or more anatomical structures (21); and• determining the accuracy and / or reliability of the prediction by running the prediction step on another machine learning network configured differently than the machine learning network (4), or by first reconfiguring the machine learning network (4) and then running the prediction step on the reconfigured machine learning network (4), and then comparing the predictions.

14. A computer program product comprising instructions for implementing the steps of the method according to any one of the preceding claims when loaded and run on a computing apparatus.

15. A data processing system (1 ) for predicting at least one axis (22), plane (24) or point (23) from medical data representing one or more anatomical structures (21), the system (1 ) being configured to:• obtain one or more 3D dataset volumes representing the medical data and comprising one or more complete or incomplete anatomical structures (21);• segment the one or more 3D dataset volumes by labelling the one or more anatomical structures (21) with a first label, and remaining elements in the one or more 3D volumes with at least a second, different label to directly or indirectly obtain a digital 3D representation (20) of the one or more anatomical structures (21 ); and• predict by using a pre-trained machine learning network (4) at least one axis (22), plane (24) and / or point (23) from the digital 3D representation (20) as processed or unprocessed, wherein the at least one axis (22), plane (24) and / or point (23) represent(s) information at least not fully contained in the digital 3D representation (20), wherein the machine learning network (4) is pre-trained on training data comprising complete or incomplete models of the one or more anatomical structures (21), and axis, plane or point information is configured to be gained during training from one or more complete models of the one or more anatomical structures (21) and configured to train the machine learning network

Images