Cardiac image analysis

Abstract

An automated method for selecting image frames from a sequence of image frames for applying one or more anatomical / physiological assessment or quantification algorithms. In particular, the method is based on classifying a received sequence of image frames according to a field of view from which the sequence of image frames was acquired. The method further comprises, for each frame in the sequence, determining a sub-phase of the image frame within a cardiac cycle of the subject's imaged heart. These determined parameters are used to identify one or more suitable frames for applying one or more predefined algorithms and / or are usable to identify one or more suitable algorithms to apply to each or a subset of the frames.

Description

[Technical field]

[0001] The present invention relates to a method for cardiac image analysis, in particular for the assessment of cardiac valves. [Background technology]

[0002] Valvular heart disease is a major class of cardiac pathology. They can be assessed using ultrasound imaging. Diagnosis usually requires lengthy examinations with manual evaluation by the clinician.

[0003] Aortic valve disease is one of the most prevalent valvular heart diseases. For example, the prevalence of aortic stenosis in developed countries is 3%.

[0004] Aortic stenosis can be assessed from ultrasound images using Doppler echocardiographic parameters such as maximum transaortic jet velocity, mean transaortic pressure gradient, or aortic valve area, which can be derived from the continuity equation.

[0005] Currently, aortic calcification is assessed using computed tomography, with the resulting Agatston score being the gold standard.

[0006] Mitral valve disease is also a major category of heart valve pathology. Mitral valve disease includes mitral stenosis or mitral regurgitation. Mitral valve disease can be evaluated using ultrasound imaging. For example, ultrasound imaging from an apical four-chamber view allows evaluation of mitral valve orifice area and allows detection of mitral stenosis. Summary of the Invention [Problem to be solved by the invention]

[0007] It has been previously proposed to perform such examinations automatically or semi-automatically using ultrasound. However, all known approaches require the user to select the appropriate frames before an automatic assessment can be applied. For example, to quantify a particular parameter, it is necessary that the ultrasound image captures the correct field of view, e.g., the parasternal short axis (PSAx) or parasternal long axis (PLAx) plane. Also, one or a series of image frames spanning a particular portion of the cardiac cycle may be required.

[0008] U.S. Patent Application Publication No. 2017 / 007201 discloses an ultrasound diagnostic device that includes a processing circuit configured to extract edges of a mitral valve based on 3D images generated by an image generation circuit, track the edges of the mitral valve, determine a timing of end-systole, and calculate an intercuspate separation based on the 3D image of a frame corresponding to end-systole.

[0009] EP 3,571,999 A1 discloses an ultrasound diagnostic apparatus including a processing circuit configured to identify a region that is part of the heart based on video data depicting the heart, and to obtain a reference waveform that allows estimating the cardiac phase based on that region in the video data. [Means for solving the problem]

[0010] The invention is defined by the claims.

[0011] According to an embodiment in accordance with one aspect of the invention, a method is provided comprising the steps of receiving a time sequence of ultrasound frames of a subject's heart over at least one cardiac cycle; determining a field of view represented by the time sequence of frames; determining for each frame a phase point of the frame within the cardiac cycle, the phase point being a sub-phase point of one of a systolic phase or a diastolic phase; applying one or more cardiac valve analysis algorithms, each cardiac valve analysis algorithm configured to receive one or more input ultrasound frames representing the heart in one or more predefined fields of view at one or more predefined phase points of the cardiac cycle, and the one or more input image frames provided to each cardiac valve analysis algorithm are selected based on the determined field of view and phase point of each of the received frames; and generating a data output representative of an output of each of the one or more cardiac valve analysis algorithms.

[0012] An embodiment of the present invention provides a method for automated cardiac valve (eg, aortic valve) assessment based on a sequence of ultrasound frames.

[0013] By detecting the field of view represented by the sequence of images and classifying each frame according to which phase interval it is in, the appropriate image frames can be automatically provided as input to the appropriate assessment algorithm, enabling heart valve assessment. More specifically, by detecting the sub-phase (which is a subpart of either systole or diastole) of each frame, it is possible to more precisely control which frames are provided to which algorithms.

[0014] The ultrasound image frames are received from an ultrasound imaging device, and the method is performed in real time with image acquisition by the imaging device. In another embodiment, the image sequence is received from a data store, in which case the method is performed offline based on image data from a prior imaging session.

[0015] The determined phase point corresponds to one of a set of predefined phase intervals, each phase interval being a subinterval of one of the systolic and diastolic phases.

[0016] In some embodiments, the predetermined phase interval includes at least a subset of the following: isovolumic relaxation, ventricular filling, atrial contraction, isovolumic contraction, and ventricular ejection.

[0017] In some embodiments, at least one of the algorithms is adapted to receive a sequence of image frames. Optionally, the method comprises selecting a temporally contiguous subset of the image frames from the received image frames based on a phase point of the frames. For example, a temporally contiguous subset of the image frames spanning one of the phase intervals is selected.

[0018] In some embodiments, the method further comprises detecting, within each frame, a motion state of one or more anatomical structures, each of the one or more anatomical structures having a periodic motion pattern over one cardiac cycle. Determining the phase points of each frame is based on the detected motion state of the one or more anatomical structures.

[0019] Determining the field of view includes classifying the field of view as one of a set of predefined fields of view, the predefined fields of view including one or more of a parasternal short axis view (PSAx), a parasternal long axis view (PLAx), an apical four-chamber view, an apical two-chamber view, and an apical long axis view.

[0020] The parasternal short-axis (PSAx) and parasternal long-axis (PLAx) views are particularly useful for examining the aortic valve. For examining the mitral valve, the apical view is particularly useful. For example, the mitral valve area can be assessed from the apical four-chamber view.

[0021] In some embodiments, at least a subset of the one or more cardiac valve analysis algorithms are adapted to output aortic and / or mitral valve parameters.

[0022] In some embodiments, the one or more cardiac valve analysis algorithms are adapted to output one or more assessment indicators of aortic stenosis (AS), aortic valve area (AVA), aortic valve calcification, mitral valve area, and mitral or aortic valve cusp number.

[0023] In some embodiments, the method further comprises receiving an ECG signal of the subject. Determining the phase points for each frame is further based on the ECG signal.

[0024] The waveform of the subject's ECG signal is closely related to the cardiac cycle. For example, the QRS pattern begins just before the ventricular contraction. The P and T waves are related to the atrial polarization and ventricular repolarization, respectively, and are therefore closely related to the cardiac phase. Therefore, if an ECG signal is available, it is provided as an additional input to the phase classifier to improve the accuracy of phase point detection.

[0025] The one or more anatomical structures detected or tracked in each image frame include the aortic valve, the mitral valve, and / or at least a portion of a ventricle or atrium of the heart.

[0026] In some embodiments, determining the phase points and / or determining the field of view is based on the use of artificial intelligence (AI) algorithms, such as machine learning algorithms.

[0027] In some embodiments, the step of determining the phase points and / or the step of determining the field of view are based on the use of a deep neural network (DNN).

[0028] In some embodiments, the step of determining the phase points and / or the step of determining the field of view are based on the use of a recurrent neural network (RNN).

[0029] Compared with classical DNNs, where only the spatial structure of an image is analyzed, RNNs and spatio-temporal networks can analyze the temporal relationship between consecutive frames in a sequence, which improves the accuracy of phase detection.

[0030] In some embodiments, the method further comprises generating a display output to cause a user interface to simultaneously display one or more of the image frames and the phase points determined for the frames.

[0031] By displaying the determined phase points for each frame, the user can assess and verify the accuracy of the determination.

[0032] Optionally, the method further comprises receiving a user input from a user interface indicating a user modification to the cardiac phase points of one or more of the image frames, in this manner the automatic phase determination can be manually overridden by a user.

[0033] In an embodiment according to a further aspect of the present invention there is provided a computer program product comprising code means configured to, when executed on a processor, cause the processor to perform a method according to any example or embodiment outlined above or below or according to any claim of the present application.

[0034] In an embodiment according to a further aspect of the invention, there is provided a processing device comprising an input / output and one or more processors adapted to: receive at the input / output a time sequence of ultrasound frames of a subject's heart over at least one cardiac cycle, determine a field of view represented by the time sequence of frames, determine for each frame a phase point of the frame in the cardiac cycle, the phase point being a sub-phase of one of a systolic phase or a diastolic phase, apply one or more cardiac valve analysis algorithms, each cardiac valve analysis algorithm configured to receive one or more input ultrasound frames representing the heart in one or more predefined fields of view at one or more predefined phase points of the cardiac cycle, and the one or more input image frames provided to each cardiac valve analysis algorithm are selected based on the determined field of view and phase point of each of the received frames, and generate a data output representative of a respective output of the one or more cardiac valve analysis algorithms.

[0035] As mentioned above, in some embodiments, the phase point corresponds to one of a set of predefined phase intervals, each phase interval being a subinterval of one of the systolic and diastolic phases.

[0036] As mentioned above, in some embodiments, the processing device is further adapted to detect a motion state of one or more anatomical structures within each frame, each of the one or more anatomical structures having a periodic motion pattern over one cardiac cycle, and determining the phase point of each frame is based on the detected motion state of the one or more anatomical structures.

[0037] An embodiment according to a further aspect of the present invention provides a system comprising a processing device according to any embodiment or example outlined above or below or as described in any claim of the present application, an ultrasound imaging device adapted to output ultrasound image data to the processing device, and a user interface including a display operably coupled to the processing device.

[0038] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0039] For a better understanding of the present invention and to show more clearly how the same may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings in which: [Brief description of the drawings]

[0040] [Figure 1] FIG. 1 illustrates steps of an exemplary method according to one or more embodiments. [Diagram 2] FIG. 1 illustrates a system including a processing device according to one or more embodiments. [Diagram 3] FIG. 3 is a diagram illustrating a sample display output showing an image frame and detected phase points. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0041] The present invention will be described with reference to the drawings.

[0042] It should be understood that the detailed description and specific examples, while illustrating exemplary embodiments of the apparatus, systems and methods of the present invention, are for illustrative purposes only and are not intended to limit the scope of the invention. These and other features, aspects and advantages of the apparatus, systems and methods of the present invention will become better understood from the following description, the appended claims and the accompanying drawings. It should be understood that the figures are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the figures to denote the same or similar parts.

[0043] According to the present invention, an automated method for selecting image frames from a sequence of image frames for applying one or more anatomical / physiological assessment or quantification algorithms is provided. In particular, the method is based on classifying a received sequence of image frames according to a field of view from which the sequence of image frames was acquired. The method further comprises, for each frame in the sequence, determining a sub-phase of the image frame within a cardiac cycle of the imaged subject's heart. These determined parameters are used to identify one or more suitable frames for applying one or more predefined algorithms and / or can be used to identify one or more suitable algorithms for applying to each or a subset of the frames.

[0044] Embodiments of the present invention are particularly applicable to the assessment of heart valves and valvular heart disease, for example the aortic or mitral valve.

[0045] An embodiment of the present invention facilitates (i) automatically classifying the imaged cardiac fields represented in an image sequence, and (ii) automatically identifying the various phases of the cardiac cycle for an ultrasound image sequence (over one or more cardiac cycles). This relieves the operator of the burden of manually performing these steps and the need to manually select suitable frames for application of one or more assessment algorithms. This results in a fully automated workflow, where an ultrasound sequence is communicated to a system or processor, and a comprehensive assessment of one or more anatomical structures, e.g., the aortic valve or the mitral valve, is generated depending on the fields represented by the sequence and the cardiac cycle phases contained in the sequence. All steps of the method are performed by a stand-alone processing device, by a computer system comprising a processing device, and / or by a processing device included in an ultrasound imaging system, e.g., a cart-based ultrasound imaging system. The steps of the method are performed online (in real time) during an ultrasound scanning session, or offline, e.g., using a computer processor.

[0046] Any of the methods described herein may be computer-implemented methods.

[0047] An advantage of certain embodiments of the present invention is that it allows for identifying various predefined subsequences within the overall sequence of images, for example isolating a subsequence of image frames that corresponds to one of the phases or sub-phases of the cardiac cycle.

[0048] FIG. 1 outlines exemplary method steps for implementation by a computer or processor in accordance with an embodiment of the present invention.

[0049] The method 10 includes receiving (12) a time sequence of ultrasound frames of a subject's heart spanning at least one cardiac cycle. This includes a sequence of 2D ultrasound images. These may be B-mode ultrasound images, for example. The sequence may span multiple cardiac cycles. The image sequence may be received from an ultrasound imaging system in real time using an ultrasound scan, or may be received from a data store.

[0050] The method 10 further includes determining 14 a field of view represented by the time sequence of frames. The field of view represented by the sequence of frames depends on the imaging position and direction (i.e., the position and orientation of the ultrasound device) used to acquire the sequence of frames, and may be defined in terms of the anatomical structures captured in the frames and the shape of those anatomical structures within the frames.

[0051] The views represented by the sequence of frames are selected from a set of predefined views, such as a set of standard cardiac views, including a parasternal short axis view (PSAx), a parasternal long axis view (PLAx), an apical four chamber view, an apical two chamber view, and an apical long axis view. Many anatomical / physiological assessments require images corresponding to specific anatomical views. For example, ultrasound-based assessment of aortic valve calcification requires parasternal short axis (PSAx) images, and calculation of aortic valve area from the continuity equation requires parasternal long axis (PLAx) images.

[0052] The method 10 further comprises determining (16), for each frame, a phase point of the frame within a cardiac cycle, the phase point being a sub-phase point of one of the systolic phase or the diastolic phase.

[0053] In some embodiments, the method 10 includes detecting a motion state of one or more anatomical structures in each frame of the sequence, each of the one or more anatomical structures having a periodic motion pattern over a cardiac cycle, and detecting the phase points is based on the detected motion state of the one or more anatomical structures. The one or more anatomical structures may include, for example, one or more heart chambers (e.g., left ventricle / right ventricle) or one or more heart valves. The motion state may include an expansion / contraction state of the anatomical structures, for example, an expansion or contraction state of a heart chamber such as the left ventricle / right ventricle. In other examples, the motion state may include a position of a heart valve such as the aortic valve or the mitral valve.

[0054] The method 10 further comprises applying (18) one or more cardiac valve analysis / assessment algorithms, each algorithm configured to receive one or more input ultrasound frames representing the heart in one or more predefined fields of view at one or more predefined phase points of the cardiac cycle. The method comprises selecting one or more input image frames to be provided to at least a subset of the algorithms based on the determined fields of view and phase points of each of the received frames. The one or more analysis or assessment algorithms are stored in a data store and retrieved for use during the method. The field of view and phase point requirements for the input image frames of each algorithm may be stored as metadata associated with each algorithm in the data store. The method comprises reading this metadata to identify the input image frame requirements of each algorithm. The one or more algorithms may be automatically triggered in response to completion of a previous step of the method.

[0055] The method further comprises generating (20) a data output representative of the output of each of the one or more cardiac valve analysis algorithms, the output being communicated to another system or device, such as a remote computer, or a local or remote data store, the communication of the data output being performed using a communications module or an input / output module.

[0056] In some embodiments, the method includes generating a display output for controlling a display device of a user interface to visually present the sequence of ultrasound image frames along with the detected phase of each image frame.

[0057] In some embodiments, the user interface allows the user to verify and / or modify the phase of each frame, for example allowing the user to adjust the time intervals between the various sub-phases, before one or more analysis or evaluation algorithms are subsequently applied, for example the method pauses until there is user input in the user interface.

[0058] As shown diagrammatically in FIG. 2, a further aspect of the present invention provides a processing device comprising an input / output (I / O) section 34 and one or more processors 36 configured to perform the method outlined above or according to any embodiment disclosed above or described below or according to any claim of the present application.

[0059] In a further aspect of the invention, a system 40 is provided comprising the processing device 32 described above and further comprising an ultrasound imaging device 42 adapted to output ultrasound image data to the processing device, this data being received at the input / output 34. The system optionally further comprises a user interface 44 comprising a display operatively coupled to the processing device.

[0060] As mentioned above, one of the steps of the method comprises determining, for each frame in the image sequence, which sub-phase of the cardiac cycle it belongs to. Details for carrying out this step will now be outlined.

[0061] The determined phase point of each frame preferably corresponds to one of a set of predefined phase intervals, each phase interval being a sub-interval of one of the systole and diastole.

[0062] Specifically, the cardiac cycle can be understood to include a sequence of five major phase intervals, each of which corresponds to a sub-phase of the cardiac cycle, and are summarized as follows:

[0063] Isovolumic relaxation: During this phase, blood flows back to the atrium.

[0064] Ventricular filling: During this phase, the atrioventricular valves open and blood begins to flow from the atria to the ventricles.

[0065] Atrial systole: The atria contract and blood continues to fill the ventricles. This marks the end of diastole.

[0066] Isovolumic contraction: The ventricles contract but the aortic and pulmonary valves remain closed. This is the beginning of systole. Blood is not yet flowing through the aorta and pulmonary artery.

[0067] Ventricular ejection: The aortic and pulmonary valves open, allowing blood to flow to the rest of the body.

[0068] The aortic valve is closed during the first four phase intervals and open during the fifth phase interval. These subphases of the cardiac cycle and the corresponding states of the aortic valve are summarized below in Table 1.

[0069] [Table 1]

[0070] The detection of phase points in each image frame can be performed using AI algorithms such as machine learning algorithms. Deep Neural Networks (DNNs) are well suited for phase point detection. However, classical image processing techniques may be used additionally or alternatively.

[0071] As mentioned above, the method comprises the step of determining, for a received image sequence, a view of the heart represented by the sequence.

[0072] Determining the field of view includes classifying the field of view as one of a set of predefined fields of view, the predefined fields of view including one or more of a parasternal short axis view (PSAx), a parasternal long axis view (PLAx), an apical four-chamber view, an apical two-chamber view, and an apical long axis view.

[0073] For example, if the method is for analysis or evaluation of an aortic valve, determining the field of view may include classifying the field of view as one of a parasternal short axis view (PSAx) and a parasternal long axis view (PLAx).If the method is for evaluation or analysis of a mitral valve, determining the field of view may include classifying the field of view as one of an apical four-chamber view, an apical two-chamber view, and an apical long axis view.

[0074] The detection of the field of view of the image sequence can be performed using an AI algorithm, such as a machine learning algorithm. For example, a neural network is applied, adapted to receive one or more of the images of the sequence as input and adapted to generate a data output indicative of a predicted field of view of the images of the sequence. In some embodiments, a deep neural network (DNN) is used. The machine learning algorithm may receive the entire image sequence or only one or a subset of the sequence of images. The latter option is preferred, since it allows the method to be applied in real time with the image acquisition, when the entire sequence of images is not available at the start of the method. Even if the entire image sequence is available, the algorithm may be trained to receive only one or a subset of the images, for example a randomly selected frame of the sequence. The machine learning algorithm is pre-trained based on a training dataset, which comprises pre-acquired image frames of the heart in different fields of view of a set of predefined standard views, each frame being manually labeled according to the field of view it represents. The machine learning algorithm can be a classifier algorithm adapted to output a classification of the input image frame according to the field of view represented in the image frame.

[0075] As mentioned above, the determination of the phase points of each of the image frames is performed by an AI algorithm, for example a machine learning algorithm. In some embodiments, the machine learning algorithm is adapted to receive as input the entire image sequence spanning one or more cardiac cycles and to output, for each frame in the sequence, a label or tag indicating the phase point to which the frame corresponds. This option improves the accuracy of the phase identification, since the algorithm can take into account the contextual information provided by the image frames before and after each frame in the sequence. In other examples, the algorithm is adapted to receive as input only a single image frame at a time and to generate an output indicating the phase point of the image frame. This latter option is preferred when applying the method in real time with image acquisition, since at the start of the invention, the entire image sequence is usually not available.

[0076] The machine learning algorithm is pre-trained based on a training dataset comprising pre-acquired image frames of the heart in different phase intervals of a predetermined set of phase intervals (each corresponding to a sub-phase of one of the systolic or diastolic phases), each frame being manually labeled according to its corresponding phase point. The machine learning algorithm may be a classifier algorithm adapted to output a classification of an input image frame according to the phase points represented in the image frame.

[0077] In one set of embodiments, the machine learning algorithm for determining the phase points can be a spatio-temporal neural network or a recurrent neural network (RNN). In this set of embodiments, the input to the algorithm is the entire sequence of image frames. Compared to classical DNNs, where only the spatial structure of the image is analyzed, RNNs and spatio-temporal neural networks can analyze the temporal relationship between successive frames of the sequence.

[0078] As mentioned above, in some embodiments, the step of determining the sub-phase points for each image frame comprises first detecting, in each frame, motion states of one or more anatomical structures, each having a periodic motion pattern over one cardiac cycle, and then determining the phase points based on the detected motion states. In some embodiments, there is a separate algorithm for detecting the motion states, followed by an algorithm for detecting the phase points based on the motion states. In some embodiments, there is an algorithm for detecting the motion states, followed by a look-up table for identifying corresponding phase points associated with the detected motion states. In some embodiments, the detection of the motion states is performed implicitly by a single machine learning algorithm as part of the step of determining the phase points for a given image frame.

[0079] In some embodiments, determining the phase points for each image frame is performed using an ECG signal of the subject. Thus, method 10 further comprises receiving an ECG signal of the subject, and determining the phase points for each frame is further based on the ECG signal.

[0080] The shape of the ECG signal is closely related to the phase of the cardiac cycle. For example, the standard QRS pattern always begins just before ventricular contraction. Furthermore, the P and T waves are associated with atrial polarization and ventricular repolarization, respectively, and are therefore closely related to cardiac phase. Therefore, if an ECG detection is available, it is optionally provided as an additional input to the phase point determination algorithm, which improves the accuracy of the phase determination, for example, based on the identification of ECG curve patterns as described above. For example, the machine learning algorithm is trained using training data entries, where the training data entries include a combination of input image frames and ECG signal data that match the time points of the frames, and each training data entry is manually tagged with a corresponding phase point of the image frame.

[0081] However, the use of an ECG is not essential, as the motion of the aortic valve and the surrounding ventricular and atrium is usually sufficient to detect the phase points, for example by machine learning algorithms.

[0082] Once the field of view represented by the image sequence has been determined and the phase point of each frame in the sequence has been determined, the method comprises applying one or more cardiac valve analysis or assessment algorithms, each algorithm being configured to receive one or more input ultrasound frames representing the heart in one or more predefined fields of view at one or more predefined phase points of the cardiac cycle. In some embodiments, the method comprises selecting one or more image frames from the sequence of image frames provided as input to at least a subset of the algorithms. At least one suitable frame or subsequence of frames is provided to each assessment algorithm, where suitable means a frame having a field of view and a phase point that matches the input requirements of the associated algorithm. In some embodiments, for each frame, the suitable one or more assessment algorithms are applied. In some embodiments, the one or more assessment or analysis algorithms are adapted to receive as input a subsequence of the sequence of image frames, for example a subsequence of consecutive image frames spanning a particular subphase of the cardiac cycle.

[0083] The assessment or analysis algorithms are for determining one or more parameters related to a heart valve, e.g., an aortic valve or a mitral valve, respectively. One or more of the algorithms are adapted to derive a quantitative measurement of the one or more parameters, such as an area of ​​the heart valve. One or more of the algorithms are adapted to derive a classification, such as the presence or absence of a particular pathological condition, or a stage of severity of a particular pathological condition.

[0084] In some embodiments, at least one of the assessment algorithms is adapted to output an indicator of the presence or severity of aortic stenosis (AS), which refers to a narrowing of the aortic valve.

[0085] In some embodiments, at least one of the assessment algorithms is adapted to output an indicator indicative of the aortic valve area (AVA). The AVA calculation can be performed directly, for example based on image analysis of the imaged aortic valve, to estimate the size of the valve. In particular, a PSAx view of the heart during the end of the systolic phase (i.e., when the aortic valve is open) can be used to achieve a direct AVA calculation. Alternatively, the AVA can be calculated indirectly using a standard AVA continuity equation, which requires inputs including the integral of the time-velocity function of the ejection jet (during systole) and the diameter of the left ventricular outflow tract (LVOT). The AVA calculation using the continuity equation can be performed using a subsequence of image frames covering the systole of the PLAx view.

[0086] In some embodiments, at least one of the assessment algorithms is adapted to output an indicator of the presence or severity of aortic valve calcification. Aortic valve calcification assessment requires one or more image frames of the PSAx (and optionally PLAx) view at end diastole.

[0087] In some embodiments, at least one of the assessment algorithms is adapted to output an indicator indicative of the mitral valve orifice area.

[0088] In some embodiments, at least one of the assessment algorithms is adapted to output an indicator indicative of the number of cusps of the mitral and / or aortic valve, for example, bicuspid aortic valve is a heart disease in which the aortic valve has only two cusps.

[0089] As noted above, in one or more embodiments, the method further comprises generating a display output for causing a user interface to simultaneously display one or more of the image frames and the determined phase points for the frames. In some examples, the user interface is controlled to provide a user control enabling selection of a desired phase point in the cardiac cycle, in response to which an image frame corresponding to the selected phase point is displayed on a display of the user interface. For example, the user interface displays a slider bar having a length, with each selectable position on the slider bar corresponding to a phase point within the entire cardiac cycle. Successive length portions of the slider bar are labeled according to the respective phase intervals of the cardiac cycle to which they correspond. In FIG. 3, a display output of an exemplary user interface is shown by way of example.

[0090] Optionally, in some embodiments, the method further comprises receiving a user input from a user interface indicating a user modification of one or more cardiac phase points of the image frame. In this manner, the automatic phase determination can be manually overridden by a user. The user interface provides user control functionality that allows selection of a given frame and input of an adjustment phase point for that frame.

[0091] The user interface includes a console having a touch panel display. Additionally or alternatively, the user interface includes a console having a display, a pointer and / or further user input means such as a keyboard.

[0092] As discussed above, some embodiments of the present invention use one or more machine learning algorithms, which are any self-training algorithms that process input data to generate or predict output data.

[0093] Machine learning algorithms suitable for use in the present invention will be apparent to those skilled in the art. Examples of suitable machine learning algorithms include decision tree algorithms and artificial neural networks. Other machine learning algorithms, such as logistic regression, support vector machines, or naive Bayes models, are suitable alternatives.

[0094] The structure of an artificial neural network (or simply neural network) is modeled on the human brain. A neural network is made up of layers, each layer containing multiple neurons. Each neuron contains a mathematical operation. Specifically, each neuron may contain different weighted combinations of a single type of transformation (e.g., the same type of transformation, sigmoid, but with different weightings). In the process of processing the input data, the mathematical operation of each neuron is performed on the input data to generate a numerical output, and the output of each layer of the neural network is fed sequentially to the next layer. The last layer provides the output.

[0095] Methods for training machine learning algorithms are well known. Typically, such methods include obtaining a training data set including training input data entries and corresponding training output data entries. An initialized machine learning algorithm is applied to each input data entry to generate a predicted output data entry. The error between the predicted output data entry and the corresponding training output data entry is used to modify the machine learning algorithm. This process may be repeated until the error converges and the predicted output data entry is sufficiently similar (e.g., ±1%) to the training output data entries. This is commonly known as a supervised learning technique.

[0096] For example, if a machine learning algorithm is formed by a neural network, the mathematical operations (weights) of each neuron can be changed until the error converges. Known methods for modifying neural networks include gradient descent, backpropagation algorithms, etc.

[0097] The above-described embodiments of the invention utilize a processing device. The processing device generally comprises a single processor or multiple processors. The processing device may be located in a single containing device, structure, or unit, or may be distributed among multiple different devices, structures, or units. Thus, a description of a processing device being adapted or configured to perform a particular step or task corresponds to the step or task being performed by any one or more of the multiple processing device elements, either alone or in combination. A person skilled in the art will understand how such a distributed processing device may be implemented. The processing device includes a communication module or input / output for receiving data and outputting data to further components.

[0098] The processor or processors of the processing device can be implemented in many ways using software and / or hardware to perform the various required functions. A processor typically utilizes one or more microprocessors, which are programmed using software (e.g., microcode) to perform the required functions. A processor may be implemented as a combination of dedicated hardware to perform some functions and one or more programmed microprocessors and associated circuitry to perform other functions.

[0099] Examples of circuitry that may be utilized in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0100] In various embodiments, the processor is associated with one or more storage media, such as volatile and non-volatile computer memories, such as RAM, PROM, EPROM, and EEPROM. The storage media is encoded with one or more programs that, when executed by the one or more processors and / or controllers, perform the necessary functions. The various storage media may be fixed within the processor or controller, or may be portable such that the processor can load one or more programs stored thereon.

[0101] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprises" does not exclude other elements or steps, and singular elements do not exclude a plurality.

[0102] A single processor or other unit may fulfill the functions of several items recited in the claims.

[0103] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0104] The computer program may be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0105] Please note that when the term "adapted to" is used in the claims or specification, it is intended to be equivalent to the term "configured to."

[0106] Any reference signs in the claims should not be construed as limiting the scope of the invention.

Claims

1. receiving a time sequence of ultrasound frames of a subject's heart over at least one cardiac cycle; determining a field of view represented by the time sequence of ultrasound frames, the field of view being an imaging field of view from which the time sequence of ultrasound frames was acquired; determining, for each ultrasound frame, a phase point of the ultrasound frame within a cardiac cycle, the phase point being a sub-phase point of one of a systolic phase or a diastolic phase; applying one or more cardiac valve analysis algorithms, each cardiac valve analysis algorithm configured to receive one or more input ultrasound frames representing the heart in one or more predefined fields of view at one or more predefined phase points of the cardiac cycle, and one or more input image frames provided to each cardiac valve analysis algorithm are selected based on the determined fields of view and phase points of each of the received input ultrasound frames; generating a data output representative of an output of each of the one or more cardiac valve analysis algorithms; The method comprising:

2. The method of claim 1 , wherein the phase points correspond to one of a set of predetermined phase intervals, each phase interval being a subinterval of one of a systolic phase and a diastolic phase.

3. The method of claim 2 , wherein the predetermined phase intervals include isovolumic relaxation, ventricular filling, atrial contraction, isovolumic contraction, and ventricular ejection.

4. 4. The method of claim 1, further comprising detecting, within each ultrasound frame, a motion state of one or more anatomical structures, each anatomical structure having a periodic motion pattern over one cardiac cycle, and wherein determining the phase points of each ultrasound frame is based on the detected motion state of the one or more anatomical structures.

5. 5. The method of claim 1, wherein determining the field of view comprises classifying the field of view as one of a set of predefined fields of view, the predefined fields of view including one or more of a parasternal short axis view (PSAx), a parasternal long axis view (PLAx), an apical four-chamber view, an apical two-chamber view, and an apical long axis view.

6. The method of claim 1 , wherein at least a subset of the one or more cardiac valve analysis algorithms are adapted to output aortic or mitral valve parameters.

7. 7. The method according to any one of claims 1 to 6, wherein the one or more cardiac valve analysis algorithms are adapted to output one or more assessment indicators of aortic valve stenosis (AS), aortic valve area (AVA), aortic valve calcification, mitral valve area, and mitral or aortic valve cusp number.

8. 8. The method of claim 1, further comprising receiving an ECG signal of the subject, and wherein the determining of the phase points for each frame is further based on the ECG signal.

9. 9. The method according to claim 1, wherein the step of determining the phase points and / or the step of determining the field of view are based on the use of an AI algorithm.

10. the step of determining the phase points and / or the step of determining the field of view are based on the use of a deep neural network (DNN); and / or The method of claim 9 , wherein the step of determining the phase points is based on the use of a recurrent neural network (RNN).

11. 11. A method according to claim 1, further comprising generating a display output for causing a user interface to simultaneously display one or more of the image frames and the phase points determined for the frames.

12. A computer program comprising code means which, when executed on a processor, causes said processor to carry out a method according to any one of claims 1 to 11.

13. A processing device comprising an input / output unit and one or more processors, said processors comprising: receiving at the input / output a time sequence of ultrasound frames of a subject's heart over at least one cardiac cycle; determining a field of view represented by the time sequence of ultrasound frames, the field of view being an imaging field of view from which the time sequence of ultrasound frames was acquired; determining, for each ultrasound frame, a phase point of the ultrasound frame within a cardiac cycle, the phase point being a sub-phase of one of a systolic phase or a diastolic phase; applying one or more cardiac valve analysis algorithms, each cardiac valve analysis algorithm receiving one or more input ultrasound frames representing the heart in one or more predefined fields of view at one or more predefined phase points of the cardiac cycle, one or more input image frames provided to each cardiac valve analysis algorithm being selected based on the determined fields of view and phase points of each of the received input ultrasound frames; and generating a data output representative of an output of each of the one or more cardiac valve analysis algorithms.

14. 14. The processing device of claim 13, wherein the phase points correspond to one of a set of predetermined phase intervals, each phase interval being a subinterval of one of a systolic phase and a diastolic phase.

15. A processing device according to claim 13 or 14, an ultrasound imaging device that outputs ultrasound image data to the processing device; a user interface operatively coupled to the processor and having a display; A system comprising:

Images