Fetal cardiac angle

Abstract

Embodiments relate to an apparatus for determining a cardiac angle of a fetus. The apparatus comprises means for obtaining an image of at least a portion of a heart and at least a portion of a chest of the fetus. The apparatus further comprises means for performing pose estimation to align the image with a standardised pose for the heart and the chest of the fetus. The means for performing pose estimation is configured to: predict a heart transformation to the image to increase an alignment between the at least a portion of the heart and the standardised pose for the heart; and predict a chest transformation to the image to increase an alignment between the at least a portion of the chest and the standardised pose for the chest. The apparatus further comprises means for determining a cardiac angle of the fetus based on the heart transformation and the chest transformation.

Description

[0001] FETAL CARDIAC ANGLE

[0002] TECHNOLOGICAL FIELD

[0003] Various example embodiments relate to an apparatus and a method for determining the cardiac angle of a fetus.

[0004] BACKGROUND

[0005] Congenital heart disease (CHD) is a common group of fetal conditions which may affect the health of a fetus. An abnormal pose of the heart within the chest may be indicative of an underlying disease. For example, a diaphragmatic hernia or spaceoccupying lesion may cause displacement, whereas an abnormal heart axis may be associated with conditions such as tetralogy of Fallot. Abnormalities of heart rhythm may also affect a fetus and can require diagnosis and treatment before birth. These can affect fetuses with a structurally normal heart.

[0006] Professional guidance, such as The American Institute of Ultrasound in Medicine (AIUM) guidelines [1] and The International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) guidelines [2], often recommends careful assessment of the cardiac position during a 20 week ultrasound screening. Conventionally, this assessment is performed in a four-chamber view which is axial to the chest. In normal anatomy, the heart is mostly on the left side of the chest with its axis tilted around 45° from the chest midline. This is known as the cardiac angle (sometimes referred to as heart angle).

[0007] A screening practitioner may measure the cardiac angle using digital callipers on a still image of the heart in the four-chamber view. However, it is often the case that only a visual check is performed due to the time required to manually perform the measurement.

[0008] It would be desirable to provide improved ways for determining the cardiac angle.

[0009] BRIEF SUMMARY

[0010] The scope of protection sought for various example embodiments of the invention is set out by the independent claims. The example embodiments and features, if any, described in this specification that do not fall under the scope of the independent claims are to be interpreted as examples useful for understanding various embodiments of the invention. According to an aspect of the invention, there is provided an apparatus for determining a cardiac angle of a fetus, the apparatus comprising: means for obtaining an image of at least a portion of a heart and at least a portion of a chest of the fetus; means for performing pose estimation to align the image with a standardised pose for the heart and the chest of the fetus, the means for performing pose estimation being configured to: predict a heart transformation to the image to increase an alignment between the at least a portion of the heart and the standardised pose for the heart; and predict a chest transformation to the image to increase an alignment between the at least a portion of the chest and the standardised pose for the heart; and means for determining a cardiac angle of the fetus based on the heart transformation and the chest transformation.

[0011] Many images obtained during, for example, an ultrasound scan are not currently used for measuring the cardiac angle because they are not in the ideal four-chamber view. Conventionally, a screening practitioner has to capture an ultrasound image in the four- chamber view and measure the cardiac angle using digital callipers on the still image of the heart in the four-chamber view. However, in practice only a visual check of the cardiac angle is performed to save time.

[0012] Embodiments propose performing pose estimation on an image by predicting a transformation to the image for aligning at least a portion of the heart shown in the image with a position of the heart within the standardised pose. Embodiments also predict a transformation to the image for aligning at least a portion of the chest shown in the image with a position of the chest within the standardised pose. The standardised pose may be defined, for example, in the four-chamber view. However, it will be appreciated that the standardised pose may also be defined in any reference view suitable for measuring the cardiac angle may be used, for example, the left ventricular outflow tract view. In some embodiments, predicting the heart transformation and the chest transformation is performed simultaneously.

[0013] The difference between the resultant heart and chest transformations provides an indication of the relative angle between the heart and the chest in the image compared to the angle of the heart and the chest in the standardised pose. Therefore, the resultant transformations may be used to determine the cardiac angle within the image.

[0014] In this way, embodiments may determine the cardiac angle even when the image does not show a standardised view, such as the four-chamber view. Accordingly, embodiments may determine the cardiac angle from images captured at imperfect positions and angles. As a result, the time needed to measure the cardiac angle and the skill required by a screening practitioner may be reduced compared to the conventional methods.

[0015] In this context, a transformation may comprise at least one of the following: translating, rotating and scaling the image. In some embodiments, the transformation comprises a rotation only. In some embodiments, the transformation additionally comprises a translation and / or a scaling. In some embodiments, the transformation comprises an affine transformation. An affine transformation may be useful for pose estimation because the heart / chest may be more closely aligned with the standardised pose. In some embodiments, the transformation comprises a deformable transformation.

[0016] Other automated techniques for measuring the cardiac angle of a fetus in development focus on classifying images which contain a suitable view for measuring the cardiac angle (e.g., the four-chamber view) and determining the cardiac angle from suitable images by detecting specific anatomical features. Like the manual techniques known in the art, these methods may still be limited to images captured from suitable standardised views. For example, the anatomical feature may not be visible in the captured image plane. Furthermore, abnormal anatomy, view instability and artifacts may hinder the ability to identify the desired anatomical feature required for calculating the cardiac angle. By comparison, embodiments use the full context of the image to align the heart and the chest with the standardised pose without relying on specific anatomical features. In this way, embodiments may be more robust to artefacts, anatomical abnormalities and view instability.

[0017] Note that the cardiac angle may sometimes be referred to as the heart angle.

[0018] In some embodiments, the image comprises a plurality of images. In this way, the cardiac angle may be determined for a number of images.

[0019] In some embodiments, the plurality of images comprises a time series of images. In some embodiments, the time series of images comprises consecutively captured images. In some embodiments, the time series of images comprises frames of a video.

[0020] In this way, the cardiac angle may be monitored over a period of time. Some congenital heart diseases are characterised by certain cardiac angles at different phases of the cardiac cycle and / or the changes to the cardiac angle throughout the cardiac cycle. Therefore, embodiments which calculate the cardiac angle multiple times over a period of time can obtain additional information useful for diagnosing congenital heart diseases.

[0021] In some embodiments, the time series of images may be analysed in real-time. In this way, embodiments may provide near real time cardiac angle measurements. In other embodiments, the time series of images are pre-recorded.

[0022] In some embodiments, the means for determining the cardiac angle is further configured to determine an average cardiac angle based on the determined cardiac angle for each image of the plurality of images.

[0023] The cardiac angle may change with time due to a regular rocking of the heart known as apical swing. It is often difficult for a screening practitioner to tell at what stage of the cardiac cycle a single image has been captured. Therefore, by calculating the average cardiac angle from a plurality of images taken at different times, a more accurate determination may be made as to whether the cardiac angle is abnormal and / or indicative of a congenital heart disease.

[0024] In some embodiments, the means for determining is configured to determine an average end-diastolic (ED) cardiac angle and / or an average end-systolic (ES) cardiac angle. In some embodiments, the means for determining is configured to determine an average cardiac angle over a whole cardiac cycle.

[0025] In some embodiments, the apparatus comprises means for calculating a heart rate of the fetus based on changes to the determined cardiac angle between each image of the plurality of images.

[0026] Conventional 2D ultrasound (b-mode) is currently used to measure cardiac angle whereas pulse-wave Doppler recording is typically used to monitor fetal heart rate. Alternatively, m-mode imaging may be used to monitor fetal heart rate when the risk of thermal bioeffects is greater during early gestation.

[0027] Switching the mode on the ultrasound probe to perform a pulse-wave Doppler recording and optimising the machine settings takes time. Furthermore, pulse-wave Doppler recordings can be challenging for many reasons including movement of the fetus disrupting the ability to calculate the heart rate.

[0028] Changes to the cardiac angle due to apical swing are indicative of the heartbeat of the fetus. Therefore, embodiments may utilise monitored changes to the cardiac angle to calculate the heart rate of the fetus. Moreover, embodiments may be more tolerant to fetal movement because the pose estimation methodology is invariant to changes in position of the fetus. As a result, embodiments may obviate the need for a pulse-wave Doppler recording.

[0029] Furthermore, by being able to monitor the heart rate, embodiments may help screen for abnormalities of heart rhythm.

[0030] The heart rate may be an average heart rate. In some embodiments, the apparatus is configured to monitor heart rate variability through continuous measurement.

[0031] In some embodiments, the means for performing pose estimation is configured to iteratively perform: the predicting the heart transformation; transforming the image using the predicted heart transformation to obtain a transformed image, wherein predicting the heart transformation is based on the transformed image in subsequent iterations; the predicting the chest transformation; and transforming the image using the predicted chest transformation to obtain another transformed image, wherein predicting the chest transformation is based on the another transformed image in subsequent iterations; wherein the cardiac angle is determined based on a final prediction of the heart transformation and a final prediction of the chest transformation.

[0032] Once the initial heart and chest transformations have been predicted, the image may be transformed using the predicted transformations. The transformed images may be used as the basis for predicting an updated heart and chest transformation. This process can be performed a number of times until a final iteration of the heart transformation and a final iteration of the chest transformation are obtained. In some embodiments, the iterative processes for predicting the heart transformation and the chest transformation are performed in parallel.

[0033] In this context, iteratively performing the prediction and transformation steps comprises performing the operations multiple times where the output transformed image of the previous iteration is used as the basis for predicting updated versions of the heart transformation and the chest transformation. Each transformation should result in an image which is more closely aligned with the standardised pose because the starting image from which the prediction is made becomes more aligned with the standardised pose.

[0034] The process of iteratively performing pose estimation may be referred to as pose correction.

[0035] In some embodiments, the means for performing pose estimation is configured to perform a predetermined number of iterations

[0036] For practical reasons, the apparatus may be limited to a certain number of iterations. If the apparatus can perform pose correction successfully for an image, the last few updates to the heart transformation and the chest transformation are likely to be minor adjustments and refinements. However, if the apparatus is unable to align the image with the standardised pose, the updates to the heart transformation and the chest transformation are likely to remain significant and possibly sporadic until the predetermined number of iterations has been reached. In some embodiments, the predetermined number is between 1 and 64. In some embodiments, the predetermined number is between 3 and 32. In some embodiments, the predetermined number is between 10 and 20. In some embodiments, the predetermined number is 16.

[0037] It will be appreciated that it may not be necessary to perform multiple iterations. The first prediction for the heart transformation and / or the chest transformation may be sufficient to align the heart and / or chest with the standardised pose, respectively with the required precision. Additional iterations will improve precision and allow uncertainty to be estimated through by observing convergence.

[0038] In some embodiments, the apparatus comprises means for generating at least one indication of an uncertainty in the determined cardiac angle based on at least one of the following: the predicted heart transformations; the predicted chest transformations; image quality; image content; and if the image comprises a plurality of images: a change in the determined cardiac angle; if the apparatus comprises means for calculating the heart rate, the heart rate; a visual similarity between two or more of the plurality of images when transformed by the heart transformation or the chest transformation; and a position of a determined cardiac angle within a distribution of the determined cardiac angle for each image of the plurality of images. It will be appreciated that embodiments may consider more than one of the parameters when calculating uncertainty.

[0039] When estimating poses iteratively (pose correction), in relation to the predicted heart transformations or the predicted chest transformations, convergence of the transformation may be an indication that the apparatus has successfully aligned the image to the standardised pose. In this case, the uncertainty in the determined cardiac angle may be low. In this context, convergence means that the updates to the transformation in the latter iterations become smaller and less severe as the transformation more accurately aligns the image to the standardised pose. Conversely, if updates to the predicted transformations do not converge, this may be an indication the apparatus is struggling or cannot align the image to the standardised pose within a reasonable number of iterations. For example, the image may not be aligned to the standardised pose when there is no heart or only a small portion of the heart shown in the image. In this case, the uncertainty in any determined cardiac angle may be high.

[0040] Image quality is influenced by many factors including artefacts, machine settings, unfavourable fetal lie and maternal habitus. Image quality may be determined in any suitable way, for example, based on contrast, motion blur, resolution, and / or image sharpness. A low image quality may mean there is a high uncertainty in the determined cardiac angle. More reliable measurements can usually be obtained from clearer images. In some embodiments, images are prefiltered to discard any images which do not meet a required quality. This may reduce the processing load.

[0041] If the image content does not contain one or more anatomical features useful for pose correction (e.g., heart apex, whole chest, spine), this may indicate that the determined cardiac angle has a higher uncertainty. More reliable measurements can usually be extracted from images showing the expected / preferred content. In some embodiments, images are prefiltered to discard any images which do not contain required content, e.g., at least a portion of the heart and at least a portion of the chest. This may reduce the processing load.

[0042] When multiple cardiac angle measurements are made for a plurality of images, a significant deviation in the determined cardiac angle between consecutively acquired images may be indicative of high uncertainty in one or more of the determined cardiac angles. For example, the change to the cardiac angle must be physically realistic. Where an unrealistic change to the cardiac angle is observed, this may be an indication of erroneous measurement and higher uncertainty. This principle may be extended to any physical change within the image in given time period, i.e. , observed changes must be physiologically realistic and so changes relating to the cardiac angle and / or other physical features / biometrics (e.g., the heart rate, chamber size) may be considered when calculating uncertainty.

[0043] Images that are dissimilar in appearance after alignment with the standardised pose are likely to be misaligned or have differing content (i.e., the heart or the chest may not be visible).

[0044] Measurements with an unusually large deviation from the observed distribution may be considered anomalies and / or have high uncertainty.

[0045] Smaller temporal groups of high certainty measurements may be rejected as they may be unreliable. Longer sequences of high certainty measurements may be preferred because they can allow the physiological plausibility to be determined.

[0046] In some embodiments, an indication of an uncertainty for other biometrics (e.g., the heart rate) may be generated. In some embodiments, this indication is generated based on the indication of the uncertainty of the cardiac angle.

[0047] In some embodiments, the apparatus comprises means for verifying that the cardiac angle has been determined for each image of the plurality of images based on the at least one indication of the uncertainty.

[0048] In this way, the apparatus may be able to verify whether the apparatus has accurately or successfully determined the cardiac angle for a sequence of images. The apparatus may provide an indication of any such sequences. In this way, the apparatus may provide an indication as to which portions of a video stream provide accurate biometric data such as cardiac angle and heart rate.

[0049] In some embodiments, changes to a cardiac angle are compared to a model for the cardiac cycle. The model may be a learnt model or an analytical model. In some embodiments, multiple models are used. Each model may pertain to a particular heart defect. The closer the fit to the model, the lower the uncertainty may be. Therefore, a verified segment / sequence of video may map closely to the model. When multiple models representing different heart defects are used, a close match to one model may indicate that a particular heart defect is present.

[0050] In some embodiments, a sliding window approach may be used in which a length of time (e.g., 3 cardiac cycles or a predetermined number of frames) is monitored at any given time. The heart rate may be continuously monitored such that a measurement of the heart rate is determined for every video frame.

[0051] In some embodiments, the means for performing pose estimation comprises a machine learning model trained to predict the heart transformation and the chest transformation.

[0052] In some embodiments, the machine learning model is trained using images of normal anatomy which are augmented to simulate abnormal anatomy.

[0053] Normal anatomy may refer to hearts and chests which are not diseased or deformed. Abnormal anatomy may refer to hearts and chests which are diseased and / or deformed.

[0054] There may be limited training data available for certain rare conditions and so artificially created training data may be generated by simulating conditions within images originally showing normal anatomy. This may mitigate the class imbalance problem and improve the model’s ability to estimate parameters more accurately in images containing diseased anatomy.

[0055] In some embodiments, the machine learning model is trained using images of normal anatomy. In some embodiments, the machine learning model is trained using images of abnormal anatomy. The images of abnormal anatomy may be real or simulated. The machine learning model may be trained using a mixture of images of normal anatomy and abnormal anatomy.

[0056] In some embodiments, the means for determining the cardiac angle is configured to calculate a relative transformation from the heart transformation and the chest transformation, the cardiac angle being determined from the relative transformation.

[0057] The heart transformation and the chest transformation may be combined through their composition to get the relative transformation between the heart and the chest. From this, the cardiac angle may be calculated. In some embodiments, the apparatus further comprises means for determining at least one of the following using at least one of the heart transformation and the chest transformation: a size of the heart; a relative position of the heart within the chest; a size of the chest; a relative size of the heart compared to the chest. These biometric parameters may be derived from at least one of the following: the heart transformation, the chest transformation, and the relative transformation calculated from the heart transformation and the chest transformation. For example, the relative transformation provides an indication of the scaling between the chest and the heart which can be used to determine their relative size.

[0058] In some embodiments, the apparatus comprises means for displaying one or more of the determined biometric parameters, e.g., the size of the heart, the cardiac angle and / or the heart rate.

[0059] In some embodiments, the apparatus comprises means for annotating the image with at least one of the following: one or more of the biometric parameters; and landmarks within the image. Annotations may include, for example: segmentations including thorax, spine, heart, four heart chambers, pulmonary veins, descending aorta, ribs; landmarks including chamber centres, heart apex, chamber centroids, septum / valves end points and biometrics such as cardiothoracic ratio, segmentation areas, landmark distances, determined angles and calculated values for certain biometric parameters. Identifying landmarks within an image may be facilitated by the pose estimation. If views other than the four-chamber view are used to define the standardised pose, other structures which are shown may be annotated such as, for example, the aortic valve and the ascending aorta, the pulmonary valve and the pulmonary artery, and the superior and inferior vena cava. The means for annotating the image may comprise a machine learning model.

[0060] The apparatus may be configured to display at least one of the following: the image, the annotated image, the image when transformed using the heart transformation, and the image when transformed using the chest transformation. Both transformed views may be complimentary. A stabilised view of the chest may allow the whole anatomy and the dynamics of the beating heart (e.g. apical swing) to be better observed without fetal motion whereas a stabilised view of the heart alone may help a clinician to better assess specific parts of the heart anatomy when the swing aspect has been removed, e.g., valve movements. In some embodiments, the means for obtaining is configured to identify from a set of images any images showing at least a portion of the heart and at least a portion of the chest, wherein the image comprises those images which show at least a portion of the heart and at least a portion of the chest.

[0061] In this way, there may be a pre-filtering step in which the apparatus performs a coarse region detection to identify images that contain both the heart and chest. This may be useful to reduce computation (e.g. when combined with a larger automated measurement system) and avoid false positive detections. This may be achieved, for example, using a machine learning model. The machine learning model may or may not be the same machine learning model trained to predict the transformations to those images which do show at least a portion of the heart and chest.

[0062] By filtering out any images which do not show the heart and the chest, the computational burden on the apparatus may be reduced. The number of false positives may also be reduced.

[0063] In some embodiments, the means for performing pose estimation is further configured to: transform each of the plurality of images using the heart transformation or the chest transformation to obtain a plurality of transformed images; and update at least one of the heart transformation and the chest transformation for at least one image of the plurality of images based on the plurality of transformed images.

[0064] The estimates for the heart transformation and / or the chest transformation may not perfectly align a group of images with the standardised pose, i.e., there may be errors for some images. Embodiments may perform a group-wise refinement to further refine the heart and chest transformations. A group of images may contain more information than each individual photo. This is because certain features may be obscured or degraded in some images but not in others. There are many factors that lead to poor quality images that contain artefacts and missing information. For example, features may be hidden by localised signal dropout caused by an inadequate acoustic window, poor probe contact, suboptimal machine settings, or strong reflectors that create shadows.

[0065] Therefore, each individual image may be incomplete whereas the group, as a whole, may show all the desired features. Embodiments may fuse the intensities of the pose corrected images to create a more detailed average template image with less noise and possibly more detail. This average template image may be used as a target for image registration. Transformation parameters may then be optimised to increase the image similarity between each image and the template image. This may include, for example, minimising an image similarity metric using a robust block matching approach. In this way, misaligned images, which are often of worse quality, may still be used to derive accurate biometric parameters by updating the heart and the chest transformations based on information retrieved from other images. In some embodiments, updating the group of heart and the chest transformations may be an iterative process. For subsequent iterations, when all transformations for the group have been updated, a new improved average template is created, before registering images to the template again. In some embodiments, a sliding window approach may be used to define groups of images for the group-wise refinement such that the computational burden on the apparatus is limited.

[0066] In some embodiments, the image comprises an ultrasound image.

[0067] In some embodiments, the means for obtaining the image comprises an imaging apparatus configured to capture the image. In some embodiments, the means for obtaining comprises means for retrieving the image from a data store. In some embodiments, the means for obtaining comprises means for receiving the image, for example, from an imaging apparatus.

[0068] In some embodiments, the standardised pose is defined in a four-chamber view of the heart and the chest of the fetus.

[0069] It will be appreciated that the standardised pose may be defined in any view of the heart and the chest suitable for determining the relative cardiac angle. The four- chamber view is one example of a suitable view. Another suitable view, for example, may be the left ventricular outflow tract view. In some embodiments, the standardised pose is defined over the whole cardiac cycle.

[0070] According to a further aspect of the invention, there is provided an apparatus, comprising: at least one processor; and at least one memory storing instructions that when executed by the at least one processor cause the apparatus at least to: obtain an image of at least a portion of a heart and at least a portion of a chest of the fetus; perform pose estimation to align the image with a standardised pose for the heart and the chest of the fetus, the pose estimation comprising: predicting a heart transformation to the image to increase an alignment between the at least a portion of the heart and the standardised pose for the heart; and predicting a chest transformation to the image to increase an alignment between the at least a portion of the chest and the standardised pose for the chest; and determine a cardiac angle of the fetus based on the heart transformation and the chest transformation.

[0071] The apparatus may be caused to perform the optional features set out in relation to an apparatus according to the aspect mentioned above.

[0072] The processor, memory, and example algorithms, encoded as instructions, program, or code, may be the means for providing or causing the performance of the operation.

[0073] According to a further aspect of the invention, there is provided a method for determining a cardiac angle of a fetus, the method comprising: obtaining an image of at least a portion of a heart and at least a portion of a chest of the fetus; performing pose estimation to align the image with a standardised pose for the heart and the chest of the fetus, the pose estimation comprising: predicting a heart transformation to the image to increase an alignment between the at least a portion of the heart and the standardised pose for the heart; and predicting a chest transformation to the image to increase an alignment between the at least a portion of the chest and the standardised pose for the chest; and determining a cardiac angle of the fetus based on the heart transformation and the chest transformation.

[0074] Further particular and preferred aspects are set out in the accompanying independent and dependent claims. Features of the dependent claims may be combined with features of the independent claims as appropriate, and in combinations other than those explicitly set out in the claims.

[0075] Where an apparatus feature is described as being operable to provide a function, it will be appreciated that this includes an apparatus feature which provides that function or which is adapted or configured to provide that function.

[0076] BRIEF DESCRIPTION

[0077] Some example embodiments will now be described with reference to the accompanying drawings in which: FIG. 1 shows an ultrasound image of a heart and a chest of a fetus in the four-chamber view;

[0078] FIG. 2 shows a block diagram of a system according to an embodiment;

[0079] FIG. 3 shows a diagram of a heart transformation and a chest transformation according to an embodiment;

[0080] FIG. 4 shows example training data for a system according to an embodiment; FIG. 5 shows a profile of a heart produced by a system according to an embodiment; FIG. 6 shows a graph comparing a determined heart angle for normal anatomy and abnormal anatomy; and

[0081] FIG. 7 shows a flow diagram illustrating steps in a method according to an embodiment.

[0082] DETAILED DESCRIPTION

[0083] Before discussing the example embodiments in any more detail, first an overview will be provided.

[0084] Congenital heart disease (CHD) is a common group of fetal conditions which may affect the health of a fetus. An abnormal pose of the heart within the chest may be indicative of an underlying disease. To screen for CHD, the angle of the heart within the chest, known as the cardiac or heart angle, may be measured. This is typically performed during a 20 week ultrasound scan and requires the screening practitioner to capture ultrasound images in a pose known as the four-chamber view. Figure 1 shows an annotated ultrasound image of normal anatomy (i.e., without defects or disease) in the four-chamber view. Figure 1 shows that the heart is mostly on the left side of the chest with a cardiac angle of 45°. Ultrasound images such as the one shown in Figure 1 are typically used by practitioners to measure the cardiac angle of the fetus. A screening practitioner may measure the cardiac angle by manually positioning digital callipers on the still image or they may perform a quick visual check to save time.

[0085] Automated tools currently in development focus on classifying ultrasound images which show the desired four-chamber view and subsequently determining the cardiac angle from those images by identifying specific anatomical features within the image.

[0086] It was recognised that existing methods for measuring the cardiac angle are limited by the view of the captured image and, in some cases, the required presence of certain anatomical features which are used to determine the cardiac angle. Embodiments propose an apparatus and a method for measuring cardiac angle which does not require an image to be captured in a standardised view such as the four- chamber view. Specifically, embodiments use a machine learning model to predict a heart transformation to align the heart shown in a captured image with the heart in a standardised pose. Concurrently, the machine learning model predicts a chest transformation to align the chest shown in the captured image with the chest in the standardised pose. Predicting the heart transformation and the chest transformation may be an iterative process in which the first predicted transformation is based on the original image. Subsequent predictions of the transformation are based on the original image transformed by the most recent prediction for the transformation. When successful, updates to the transformations become smaller and less significant. In some embodiments, the process is repeated a predetermined number of times. The relative difference between the predicted heart transformation and the predicted chest transformation provides an indication of the relative angle between the heart and the chest which is used to determine the cardiac angle.

[0087] In this way, embodiments may determine the cardiac angle even when the captured image is not in a standardised view such as the four-chamber view. Moreover, embodiments may make use of the full context of the image to align the heart and the chest to their respective standardised poses. Therefore, embodiments may not rely on specific anatomical features to be present potentially making embodiments more robust to artefacts, anatomical abnormalities and view instability compared to other automated methods for determining the cardiac angle.

[0088] In some embodiments, the cardiac angle is determined for multiple images captured over a period of time. Since the cardiac angle of a fetus changes throughout the cardiac cycle, embodiments may calculate the heart rate of a fetus based on the monitored cardiac angle. Conventionally, measuring the heart rate of a fetus is performed using a pulse-wave Doppler recording. However, pulse-wave Doppler recordings can be challenging for many reasons including movement of the fetus disrupting the ability to determine the heart rate. Embodiments do not require use of a pulse-wave Doppler recording to calculate the heart rate and so may not suffer from the same drawbacks.

[0089] Figure 2 shows a block diagram of a system 1 according to an embodiment. At the image acquisition stage, the system 1 obtains at least one image showing at least a portion of a heart and at least a portion of a chest of the fetus. The image may, for example, be an ultrasound image. However, it will be appreciated that any image type which enables a cardiac angle to be determined may be used. In some embodiments, the at least one image comprises a plurality of images. The images may be frames of a real-time video stream or a prerecorded video clip. The images may be consecutive image frames. In some embodiments, the system 1 processes the images of a whole examination in real-time or at a later time.

[0090] In some embodiments, the system 1 is configured to identify from an input set of images any images showing at least a portion of the heart and at least a portion of the chest. Only those images which are determined to show at least a portion of the heart and at least a portion of the chest are subsequently processed. Pre-filtering images in this way may avoid unnecessarily images from which it is not possible to determine the cardiac angle from. The filtering may be performed by a machine learning model trained to classify images as either showing the desired content or not.

[0091] At the pose estimation stage, the system 1 performs an iterative process to estimate a heart transformation and a chest transformation for each image to be processed. The transformations may comprise translating, rotating and scaling the image. The heart transformation aims to align the at least a portion of the heart shown in an image to the standardised pose for the heart. Similarly, the chest transformation aims to align the at least a portion of the chest shown in the image with the standardised pose for the chest. The standardised pose may be defined in any view of the heart and the chest which enables the cardiac angle to be determined including but not limited to commonly captured views such as the four-chamber view or the left ventricular outflow tract.

[0092] To obtain the heart transformation, the system loops through the steps:

[0093] 1. Transform an original image using the current heart transformation (initially, the current transformation is the identity, i.e., no transformation);

[0094] 2. Predict a transformation update for the heart transformation; and

[0095] 3. Update the current heart transformation estimate using the predicted transformation update.

[0096] The same steps are repeated to obtain the chest transformation: 1. Transform the original image using the current chest transformation (initially, the current transformation is the identity, i.e., no transformation);

[0097] 2. Predict a transformation update for the chest transformation; and

[0098] 3. Update the current chest transformation estimate using the predicted transformation update.

[0099] In other words, the system 1 is configured to iteratively perform: predicting a heart transformation to the image to increase an alignment between the at least a portion of the heart and the standardised pose for the heart; and transforming the image using the heart transformation to obtain a transformed image, wherein predicting the heart transformation is based on the transformed image in the next iteration.

[0100] Similarly, the system is configured to iteratively perform: predicting a chest transformation to the image to increase an alignment between the at least a portion of the chest and the standardised pose for chest; and transforming the image using the chest transformation to obtain another transformed image, wherein predicting the chest transformation is based on the another transformed image in the next iteration.

[0101] Performing pose estimation by iteratively updating the predicted heart transformation and the predicted chest transformation may be referred to as pose correction.

[0102] In some embodiments, pose estimation is performed for the heart and the chest concurrently.

[0103] In some embodiments, the system 1 comprises a machine learning model trained to predict the heart transformation and the chest transformation. Specifically, in some embodiments, the machine learning model is trained to predict updates to the heart transformation and the chest transformation for a given input image. The pose estimation may be performed automatically in real-time.

[0104] In some embodiments, the system 1 is configured to perform a predetermined number of iterations when estimating the heart transformation and the chest transformation. The apparatus may be limited to a certain number of iterations for practical reasons. If the system 1 is able to perform pose correction successfully, the last few updates to the heart transformation and the chest transformation are likely to be minor adjustments. However, if the apparatus is unable to perform pose correction, the updates to the heart transformation and the chest transformation are likely to remain significant until the predetermined number of iterations has been reached. In some embodiments, the predetermined number is between 1 and 64. In some embodiments, the predetermined number is between 3 and 32. In some embodiments, the predetermined number is between 10 and 20. In some embodiments, the predetermined number is 16.

[0105] It will be appreciated that, in some embodiments, pose estimation may not be an iterative process. In this case, pose estimation comprises predicting the heart transformation to increase an alignment between the at least a portion of the heart and the standardised pose for the heart and predicting the chest transformation to increase an alignment between the at least a portion of the chest and the standardised pose for the chest. There is no need to perform the transformations on the image because there are no subsequent predictions being made and so no transformed images are required as basis for those predictions. The cardiac angle is determined from the initial predictions for the heart and chest transformations.

[0106] The first prediction for the heart transformation and / or the chest transformation may be sufficient to align the heart and / or chest with the standardised pose, respectively, with the required precision. Additional iterations may improve precision and allow uncertainty to be estimated by observing convergence. In some embodiments, a first best effort prediction for the heart and the chest transformations is used to determine the cardiac angle. This may reduce the required processing power of the system 1 and reduce the time taken to determine the cardiac angle.

[0107] In some embodiments, the system 1 is configured to perform: transforming each of the plurality of images using the final iteration of the heart transformation or chest transformation; and updating the heart transformation and the chest transformation for at least one image of the plurality of images based on the plurality of images when transformed using the final iteration of the heart transformation or chest transformation.

[0108] The estimates for the final iteration of the heart transformation or chest transformation may not perfectly align a group of images with the standardised pose, i.e., there may be errors for some images. Embodiments may perform a group-wise refinement to further refine the heart and chest transformations. A group of images may contain more information than each individual photo. This is because certain features may be obscured or degraded in some images but not in others. There are many factors that lead to poor quality images that contain artefacts and missing information. For example, features may be hidden by localised signal dropout caused by an inadequate acoustic window, poor probe contact, suboptimal machine settings, or strong reflectors that create shadows. Therefore, each individual image may be incomplete whereas the group, as a whole, may show all the desired features. Embodiments may fuse the intensities of the pose corrected images to create a more detailed average template image with less noise and possibly more detail. This average template image may be used as target for image registration. Transformation parameters may then be optimised to increase the image similarity between each image and the template image. This may include, for example, minimising an image similarity metric using a robust block matching approach. In this way, misaligned images, which are often of worse quality, may still be used to derive accurate biometric parameters by updating the heart and the chest transformations based on information retrieved from other images. In some embodiments, updating the group of heart and chest transformations may be an iterative process. For subsequent iterations, when all transformations for the group have been updated, a new improved average template may be created, before registering images to the template again. In some embodiments, a sliding window approach may be used to define groups of images for the group-wise refinement such that the computational burden on the system 1 is limited.

[0109] Figure 3 shows a simplified illustration of the heart and the chest transformations. The top left quadrant represents an original image showing a heart 6 within a chest 8. The standardised pose of a heart and a chest with an expected cardiac angle is shown in the bottom left quadrant. A single step or an iterative process as described above can be used to estimate a heart transformation which when applied to the original image will align the heart of the original image with its standardised pose. As shown in the top right quadrant, the heart of the transformed image is aligned with its standardised pose. Note that the chest of the transformed image remains at the same angle relative to the heart as in the original image in the top left quadrant. Therefore, when the heart is transformed to align with the standardised pose for the heart, the chest will not be aligned with the standardised pose for the chest. Similarly, a single step or an iterative process as described above can be used to estimate a chest transformation which when applied to the original image will align the chest of the original image with its standardised pose. As shown in the bottom right quadrant, the chest of the transformed image is aligned with its standardised pose. Note that the heart of the transformed image remains at the same angle relative to the chest as in the original image in the top left quadrant. Therefore, when the chest is transformed to align with the standardised pose for the chest, the heart will not be aligned with the standardised pose for the heart.

[0110] In the instantaneous biometry stage, the system 1 is configured to calculate the cardiac angle within an image based on the final iteration of the heart transformation and the final iteration of the chest transformation. The system 1 may compose the two final transformation estimates for the heart and the chest to obtain a relative transformation describing the relationship between the heart and the chest in the image.

[0111] Based on the relative transformation the cardiac angle can be determined. This is because the cardiac axis in the heart standardised pose is aligned with the anterior- posterior axis in the chest standardised pose and the relative transformation describes the rotation between the two axes.

[0112] In some embodiments, when multiple images are captured, the system 1 may be configured to determine an average cardiac angle based on the determined cardiac angle for each image of the plurality of images. In some embodiments, the average cardiac angle is determined over a predetermined period of time such as one cardiac cycle.

[0113] In some embodiments, the system may be configured to determine other useful biometric parameters, such as one or more of the following: a size of the heart; a relative position of the heart within the chest; a size of the chest; and a relative size of the heart compared to the chest. The biometric parameters may be determined based on the original image when transformed using one of the heart transformation or the chest transformation and / or the difference between the estimated heart and chest transformations (the relative transformation).

[0114] The additional biometrics may be extracted by automatically annotating key landmarks or tissues. In some embodiments, a machine learning model may be used to determine the one or more biometric parameters. The additional biometrics not only adds to the useful biometrics determined by the system 1 for screening purposes but may also help the system 1 to be more confident about other measurements by building a more detailed profile of the heart. For example, in the four-chamber view, where changes in chamber area correlate with the change in heart angle, the system may be more confident in its detection of the beating heart and make more accurate predictions of the heart rate. In some embodiments, the system 1 may be configured to display for the screening practitioner one or more of the determined biometric parameters, e.g., the size of the heart and / or the cardiac angle.

[0115] The system 1 may be configured to display at least one of the following: the original image, the annotated image, the image when transformed using the final iteration of the heart transformation (a pose corrected image), and the image when transformed using the final iteration of the chest transformation (a pose corrected image). Both transformed views may be complimentary. A stabilised view of the chest may allow the whole anatomy and the dynamics of the beating heart (e.g. apical swing) to be better observed without fetal motion whereas a stabilised view of the heart alone may help a clinician to better asses specific parts of the heart anatomy when the swing aspect has been removed, e.g., valve movements. When a video is being processed by the system, a continuous stream of one or more of these images may be displayed. For example, a continuous stream of pose corrected images may be displayed.

[0116] In some embodiments, the system 1 is configured to annotate the image with at least one of the following: one or more of the biometric parameters and landmarks within the image. Annotations may include, for example: thorax, spine, heart, four heart chambers, pulmonary veins, descending aorta, ribs, chamber centres, heart apex, chamber centroids, septum / valve end points and biometrics such as cardiothoratic ratio, segmentation areas, landmark distances, determined angles and calculated values for certain biometric parameters. Identifying landmarks within an image may be facilitated by the pose correction and may be identified by a machine learning model.

[0117] In the time series biometry stage, biometric parameters which require measurement over a period of time may be determined based on the instantaneous biometric parameters determined by the system over a period of time. For example, the system may be configured to calculate a heart rate of the fetus based on changes to the determined cardiac angle over time (i.e., apical swing). The instantaneous heart rate may be displayed or annotated on an image being displayed.

[0118] In this way, embodiments may measure the heart rate without use of a conventional pulsed-wave Doppler imaging. This may save time during the examination, reduce technical acquisition errors, streamline scanning (because, in some embodiments, measurement of the heart rate may happen in the background during a usual four chamber assessment) and reduce cognitive distraction from task-switching leaving the sonographer to focus on the content of the scan instead of optimising the Doppler acquisition. Moreover, embodiments may provide a continuous measurement of the heart rate compared to a single measurement obtained by the Doppler, thereby allowing for an improved assessment of heart rate-variability. Additionally, embodiments may be more reliable as Doppler acquisition may be disrupted by fetal breathing, fetal movement and maternal breathing which can lead to poor or complete loss of signal.

[0119] Further, embodiments may be particularly useful in the first trimester during which pulsed-wave Doppler is not useable due to concerns of thermal bioeffects. Instead, m- mode imaging is typically employed during this period which can pose challenges in interpretation making precise calliper placement difficult. Small inaccuracies in calliper placement can result in significant deviations in the recorded heart rate. As a result, embodiments may calculate the heart rate more reliably.

[0120] In some embodiments, the system is configured to determine the cardiac phase from the oscillating heart angle. In some embodiments, the system 1 is configured to aggregate measurements of cardiac biometrics over multiple cardiac cycles, e.g., an average heart angle, an average end-diastolic (ED) heart angle, an average end- systolic (ES) heart angle, an average heart rate and heart rate variability. Averaged biometrics are less variable than a single measurements and can be more useful as predictive biomarkers.

[0121] At the uncertainty estimation stage, the system 1 is configured to generate at least one indication of an uncertainty in the determined cardiac angle based on at least one of the following: the predicted heart transformations; the predicted chest transformations; image quality; image content; a change in the determined cardiac angle; the heart rate; a visual similarity between two or more of the plurality of images when transformed by the final iteration of the heart transformation or chest transformation; and a position of a determined cardiac angle within a distribution of the determined cardiac angle for each image of the plurality of images. It will be appreciated that embodiments may consider more than one of the parameters when calculating uncertainty.

[0122] In relation to the predicted heart transformations or the predicted chest transformations, convergence on a stable transformation estimate may be an indication that the apparatus has successfully aligned the image to the standardised pose. In this case, the uncertainty in the determined cardiac angle may be low. In this context, convergence means that the updates to the transformation in the latter iterations become smaller and less severe as the transformation more accurately aligns the image to the standardised pose. Conversely, if updates to the predicted transformations does not converge, this may be an indication the apparatus is struggling or cannot align the image to the standardised pose within a reasonable number of iterations. For example, it may not be possible to align the image with the standardised heart pose when there is no heart or only a small portion of the heart shown in the image. In this case, the uncertainty in any determined cardiac angle may be high.

[0123] Image quality is influenced by many factors including artefacts, machine settings, unfavourable fetal lie and maternal habitus. Image quality may be determined in any suitable way, for example, based on contrast, motion blur, resolution, and / or image sharpness. A low image quality may mean there is a high uncertainty in the determined cardiac angle. More reliable measurements can usually be obtained from clearer images. In some embodiments, images are prefiltered to discard any images which do not meet a required quality. This may reduce the processing load.

[0124] If the image content does not contain one or more anatomical features useful for pose correction (e.g., heart apex, whole chest, spine), this may indicate that the determined cardiac angle has a higher uncertainty. More reliable measurements can usually be extracted from images showing the expected / preferred content. In some embodiments, images are prefiltered to discard any images which do not contain required content, e.g., at least a portion of the heart and at least a portion of the chest. This may reduce the processing load.

[0125] When multiple cardiac angle measurements are made for a plurality of images, a significant deviation in the determined cardiac angle between images may be indicative of high uncertainty in one or more of the determined cardiac angles. For example, the change to the cardiac angle must be physically realistic. Where an unrealistic change to the cardiac angle is observed, this may be an indication of erroneous measurement and higher uncertainty. This principle may be extended to any physical change within the image in given time period, i.e. , observed changes must be physiologically realistic and so changes relating to the cardiac angle and / or other physical features / biometrics (e.g., the heart rate) may be considered when calculating uncertainty. Images that are dissimilar in appearance after pose correction are likely to be misaligned or have differing content (i.e. , the heart or the chest may not be visible).

[0126] Measurements with an unusually large deviation from the observed distribution may be considered anomalies and / or have high uncertainty. Small temporally isolated groups of otherwise high certainty measurements may be considered unreliable.

[0127] In some embodiments, an indication of an uncertainty for other biometrics (e.g., the heart rate) may be generated. In some embodiments, this indication is generated based on the indication of the uncertainty of the cardiac angle. In some embodiments, an indication of an overall uncertainty based on all measured biometric parameters may be generated.

[0128] The calculation of an indication of uncertainty in measurements made by the system 1 may allow clinicians to be more confident about the automated assessment.

[0129] Typically, machine learning model uncertainty estimation requires significant computation (e.g. running many models to get a predictive distribution) and so real-time deployment may be unfeasible where computing resources are constrained. Segmentation models used by some automated cardiac angle measurement tools often require greater resources with encoder and decoder pathways and additional post-processing compared to the pose correction model implemented in some embodiments which is a regression network that requires only an encoding branch and runs at a relatively low resolution making it computationally efficient. This may facilitate deployment of the system 1 including an uncertainty estimation for real-time scenarios.

[0130] At the temporal model fit stage, the system may generate a distinctive profile of the heart that characterises its form and function and is built from many biometric measurements over time (which may be coupled with respective uncertainty information). In some embodiments, the system is configured to measure the discrepancy of this observed profile compared to a prior temporal model (which captures the dynamics of the beating heart, e.g., apical swing, chamber contraction etc). In this way, the system may be able to assess the plausibility of the observed profile, i.e., is the determined profile physiologically realistic. From this profile, the system may automatically detect sections of the examination in which the images captured by the sonographer maintain a suitable view of the heart for sufficient time to allow for a reliable measurement of the time series biometrics. Additionally, in some embodiments, when a satisfactory sequence of frames has been obtained, this information may be fed back to the sonographer during the examination. An example profile is shown in Figure 5.

[0131] In other words, the system 1 may be configured to verify that a cardiac angle (and / or other biometric parameters) has been successfully determined for each image of a plurality of images to a required confidence level. This may be based on the uncertainty for a respective biometric parameter or an uncertainty for a combination of biometric parameters being below a threshold requirement. The system 1 may provide an indication to a screening practitioner as to which portions of a video have been used to obtain accurate biometric data such as cardiac angle and / or heart rate.

[0132] In some embodiments, changes to a cardiac angle are compared to a model for the cardiac cycle. The model may be a learnt model or an analytical model. In some embodiments, multiple models are used. Each model may pertain to a particular heart defect. The closer the fit to the model, the lower the uncertainty in the cardiac angle measurements may be. Therefore, a verified segment of video (i.e., a sequence of frames used to obtain measurements of the cardiac angle with a high confidence) may map closely to the model. When multiple models representing different heart defects are used, a close match to one model may indicate that a particular heart defect is present.

[0133] In some embodiments, a sliding window approach may be used in which a length of time (e.g., 3 cardiac cycles or a predetermined number of frames) is processed at any given time.

[0134] In some embodiments, the system 1 processes an entire examination in real-time, enabling immediate feedback to the sonographer on cardiac parameters during image acquisition. This may help ensure that recordings and measurements of the required quality have been achieved during the examination.

[0135] The system 1 may also be beneficial because it may provide explicit additional information in the form of cardiac biometrics that may not usually be measured in current practice or are only visually assessed due to time constraints. Ultimately, this may increase detection rates of CHD. Another advantage of the system 1 is that the heart anatomy is aligned to a uniform pose. Maintaining a consistent pose across the image stream may give a strong prior for tissue location and empowers spatio-temporal annotation machine learning models to discern the evolving appearance of structures over time. This may facilitate accurate tracking of tissues and ensure annotation consistency.

[0136] Another advantage of pose correction is that it can enable the use of spatiotemporal shape analysis of annotated structures to further uncover abnormal function and morphology.

[0137] Furthermore, the system 1 may be more robust in determining the cardiac angle because it has the ability to use any contextual information to estimate the cardiac and chest poses whereas other automated methods typically rely on segmenting specific parts of the anatomy which may not be clearly observed due to imaging artefacts, disease or the field of view. For example, in [3] the cardiac axis is extracted by fitting a line to the segmented ventricular septum, whereas in [4] the cardiac axis is found by detecting the septum endpoints. These features are only visible in the four-chamber view and may not be reliably extracted in cases of CHD with septal defects, one of the most common forms of CHD.

[0138] Another related advantage of the system 1 is the ability to estimate the cardiac axis from a wider range of views, e.g., neighbouring axial views such as the left ventricular outflow-tract (LVOT) whereas other known methods for determining cardiac angle are inherently limited to the four-chamber view where the septum and four chambers are visible. This means the system 1 may more robustly extract uninterrupted biometrics over time even in the presence of motion and enhances its utility for monitoring cardiac function over an extended period.

[0139] The system may comprise an apparatus including any means necessary for carrying out the steps of the embodiments discussed herein. The apparatus may comprise circuitry for performing the steps of the embodiments discussed herein. In Figure 2, the apparatus comprises at least one processor 2 and at least one memory 4 storing instructions that when executed by the at least one processor 2 cause the apparatus at least to perform the steps of the embodiments discussed herein.

[0140] Figure 4a shows an example of an ultrasound image of normal anatomy. Figures 4b and 4c show an examples of an ultrasound images in which parts of the septum have been removed to simulate abnormal / diseased anatomy. The machine learning model trained to predict the heart transformation and the chest transformation may be trained using images of normal anatomy such as that shown in Figure 4a. Additionally or alternatively, the machine learning model may be trained using images of abnormal anatomy. These may be real images of abnormal anatomy or images of normal anatomy which are augmented to simulate abnormal anatomy. It is noted that normal anatomy may refer to hearts and chests which are not diseased or deformed. Abnormal anatomy may refer to hearts and chests which are diseased and / or deformed.

[0141] A common challenge for medical imaging machine learning models is diminished performance when attempting to predict or estimate parameters within rare diseased cohorts. Since there is relatively little abnormal data to learn from, a model may struggle to accurately represent and predict its characteristics. This issue is of particular importance, as these parameters often serve as key identifiers for individuals within these diseased populations. By augmenting the dataset with simulated cases of abnormal anatomy, the class imbalance problem may be addressed, thereby improving the model's ability to accurately determine biometric parameters within affected groups of abnormal anatomy.

[0142] One complication is that the cardiac axis may not be well defined in some types of CHD, e.g., where the ventricular septum is missing or displaced. This can make accurate annotation challenging. The system 1 may circumvent this problem by reusing annotations on normal anatomy with augmented anatomy that simulates disease, e.g., where the septum has been artificially removed.

[0143] Figure 5 shows a graph of a profile generated by the system 1. The heart angle is automatically measured by the system 1 for a short video clip from an ultrasound examination. The region where the sonographer is focussed on the heart is detected by the system 1 and is represented by shaded region 11. Where the heart is observed later in the clip, the green line 12 shows low pose deviation (good convergence) and the yellow line 13 shows a high probability that the correct anatomy is present, both of which indicate high certainty measurements. The detected heart rate for the dotted area is shown in the top left with an accompanying model fit score which provides an indication of the confidence / certainty in the measurements. The blue line 14 shows the measured cardiac angle. Figure 5 shows that the cardiac angle loosely follows a sine wave pattern during the period 11. Hence, a sine wave can be used as a simple model for the changes to the cardiac angle. Figure 6 shows a graph of the population distributions for the average cardiac angle (aggregated over the cardiac cycle) for a normal cohort 15 and a cohort with atrioventricular septal defects (AVSD) 16, a type of CHD. It shows how the cardiac angle may provide an indication of a CHD.

[0144] Figure 7 illustrates steps in a method according to an embodiment.

[0145] In step S1 , the method comprises obtaining an image of at least a portion of a heart and at least a portion of a chest of the fetus.

[0146] Steps S2 and S3 are, in this example embodiment, performed iteratively and comprise performing pose estimation to align the image with a standardised pose for the heart and the chest of the fetus. The pose estimation comprises iteratively performing: predicting a heart transformation to the image to increase an alignment between the at least a portion of the heart and the standardised pose for the heart; and transforming the image using the heart transformation to obtain a transformed image, wherein predicting the heart transformation is based on the transformed image in subsequent iterations. The transforming further comprises iteratively performing: predicting a chest transformation to the image to increase an alignment between the at least a portion of the chest and the standardised pose for the chest; and transforming the image using the chest transformation to obtain another transformed image, wherein predicting the chest transformation is based on the another transformed image in subsequent iterations. Estimating the chest transformation and the heart transformation may be performed simultaneously.

[0147] Specifically, step S2 comprises obtaining the heart transformation by looping through the steps:

[0148] S2a Transform an original image using the current heart transformation (initially, the current transformation is the identity, . i.e. , no transformation);

[0149] S2b Predict a transformation update for the heart transformation; and

[0150] S2c Update the current heart transformation estimate for the heart using the predicted transformation update.

[0151] The same steps are repeated in S3 to obtain the chest transformation: S3a Transform the original image using the current chest transformation (initially, the current transformation is the identity, . i.e. , no transformation);

[0152] S3b Predict a transformation update for the chest transformation; and

[0153] S3c Update the current chest transformation estimate for the heart using the predicted transformation update.

[0154] At the end of steps S2 and S3, two final transformation estimates have been generated - one for the chest and one for the heart.

[0155] Step S4 comprises determining a cardiac angle of the fetus based on the final iterations of the heart transformation and the chest transformation.

[0156] A person of skill in the art would readily recognize that steps of various abovedescribed methods can be performed by programmed computers. Herein, some embodiments are also intended to cover program storage devices, e.g., digital data storage media, which are machine or computer readable and encode machineexecutable or computer-executable programs of instructions, wherein said instructions perform some or all of the steps of said above-described methods. The program storage devices may be, e.g., digital memories, magnetic storage media such as a magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media. The embodiments are also intended to cover computers programmed to perform said steps of the above-described methods. The term non-transitory as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g. RAM vs ROM).

[0157] As used in this application, the term “circuitry” may refer to one or more or all of the following:

[0158] (a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry) and

[0159] (b) combinations of hardware circuits and software, such as (as applicable):

[0160] (i) a combination of analog and / or digital hardware circuit(s) with software / firmware and

[0161] (ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions) and (c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0162] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware.

[0163] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

[0164] The ordering of method steps set out above may not be critical or fixed and the exact ordering of the steps may be varied as appropriate.

[0165] Although example embodiments of the present invention have been described in the preceding paragraphs with reference to various examples, it should be appreciated that modifications to the examples given can be made without departing from the scope of the invention as claimed.

[0166] Features described in the preceding description may be used in combinations other than the combinations explicitly described.

[0167] Although functions have been described with reference to certain features, those functions may be performable by other features whether described or not.

[0168] Although features have been described with reference to certain embodiments, those features may also be present in other embodiments whether described or not.

[0169] Whilst endeavouring in the foregoing specification to draw attention to those features of the invention believed to be of particular importance it should be understood that the Applicant claims protection in respect of any patentable feature or combination of features hereinbefore referred to and / or shown in the drawings whether or not particular emphasis has been placed thereon. References

[0170] [1] (2020), AIUM Practice Parameter for the Performance of Fetal Echocardiography. J Ultrasound Med. 39(1), e5-e16. https: / / doi.org / 10.1002 / jum.15188

[0171] [2] Carvalho et al. (2023). ISUOG Practice Guidelines (updated): fetal cardiac screening. UOG, 61(6), 788-803. https: / doi.org / 10.1002 / uog.26224

[0172] [3] Arnaout et al. (2018). Deep-learning models improve on community-level diagnosis for common congenital heart disease lesions https: / / arxiv.org / abs / 1809.06993

[0173] [4] Krishnan K., inventor; Koninklijke Philips NV, assignee, Methods and systems for fetal heart assessment. EP4106633B1.

Claims

CLAIMS1. An apparatus for determining a cardiac angle of a fetus, the apparatus comprising: means for obtaining an image of at least a portion of a heart and at least a portion of a chest of the fetus; means for performing pose estimation to align the image with a standardised pose for the heart and the chest of the fetus, the means for performing pose estimation being configured to: predict a heart transformation to the image to increase an alignment between the at least a portion of the heart and the standardised pose for the heart; and predict a chest transformation to the image to increase an alignment between the at least a portion of the chest and the standardised pose for the chest; and means for determining a cardiac angle of the fetus based on the heart transformation and the chest transformation.

2. An apparatus according to claim 1, wherein the image comprises a plurality of images.

3. An apparatus according to claim 2, wherein the plurality of images comprises a time series of images.

4. An apparatus according to claim 2 or claim 3, wherein the means for determining the cardiac angle is further configured to determine an average cardiac angle based on the determined cardiac angle for each image of the plurality of images.

5. An apparatus according to any one of claims 2 to 4, wherein the apparatus comprises means for calculating a heart rate of the fetus based on changes to the determined cardiac angle between each image of the plurality of images.

6. An apparatus according to any preceding claim, wherein the means for performing pose estimation is configured to iteratively perform: the predicting the heart transformation;transforming the image using the predicted heart transformation to obtain a transformed image, wherein predicting the heart transformation is based on the transformed image in subsequent iterations; the predicting the chest transformation; and transforming the image using the predicted chest transformation to obtain another transformed image, wherein predicting the chest transformation is based on the another transformed image in subsequent iterations; wherein the cardiac angle is determined based on a final prediction of the heart transformation and a final prediction of the chest transformation.

7. An apparatus according to claim 6, wherein the means for performing pose estimation is configured to perform a predetermined number of iterations.

8. An apparatus according to any preceding claim, wherein the apparatus comprises means for generating at least one indication of an uncertainty in the determined cardiac angle based on at least one of the following: the predicted heart transformations; the predicted chest transformations; image quality; image content; and if the image comprises a plurality of images: a change in the determined cardiac angle; if the apparatus comprises means for calculating the heart rate, the heart rate; a visual similarity between two or more of the plurality of images when transformed by the heart transformation or the chest transformation; and a position of a determined cardiac angle within a distribution of the determined cardiac angle for each image of the plurality of images.

9. An apparatus according to claim 8 when dependent on claim 2, wherein the apparatus comprises means for verifying that the cardiac angle has been determined for each image of the plurality of images based on the at least one indication of the uncertainty.

10. An apparatus according to any preceding claim, wherein the means for performing pose estimation comprises a machine learning model trained to predict the heart transformation and the chest transformation.

11. An apparatus according to claim 10, wherein the machine learning model is trained using images of normal anatomy which are augmented to simulate abnormal anatomy.

12. An apparatus according to any preceding claim, wherein the means for determining the cardiac angle is configured to calculate a relative transformation from the heart transformation and the chest transformation, the cardiac angle being determined from the relative transformation.

13. An apparatus according to any preceding claim, wherein the apparatus further comprises means for determining at least one of the following using at least one of the heart transformation and the chest transformation: a size of the heart; a relative position of the heart within the chest; a size of the chest; a relative size of the heart compared to the chest.

14. An apparatus according to any preceding claim, wherein the means for obtaining is configured to identify from a set of images any images showing at least a portion of the heart and at least a portion of the chest, wherein the image comprises those images which show at least a portion of the heart and at least a portion of the chest.

15. An apparatus according to claim 2 or any one of claims 3 to 14 when dependent on claim 2 , wherein the means for performing pose estimation is further configured to: transform each of the plurality of images using the heart transformation or the chest transformation to obtain a plurality of transformed images; and update at least one of the heart transformation and the chest transformation for at least one image of the plurality of images based on the plurality of transformed images.

16. An apparatus according to any preceding claim, wherein the image comprises an ultrasound image.

17. An apparatus according to any preceding claim, wherein the standardised pose is defined in a four-chamber view of the heart and the chest of the fetus.

18. A method for determining a cardiac angle of a fetus, the method comprising: obtaining an image of at least a portion of a heart and at least a portion of a chest of the fetus; performing pose estimation to align the image with a standardised pose for the heart and the chest of the fetus, the pose estimation comprising: predicting a heart transformation to the image to increase an alignment between the at least a portion of the heart and the standardised pose for the heart; and predicting a chest transformation to the image to increase an alignment between the at least a portion of the chest and the standardised pose for the chest; and determining a cardiac angle of the fetus based on the heart transformation and the chest transformation.

Images