Image sequence analysis

Abstract

The present invention relates to a system and method for achieving more accurate results when applying an image processing task to a series of medical images of a patient without significantly increasing processing resources. The proposed system and method is based on receiving a plurality of image sequences of a specific anatomical region, each image sequence capturing a periodic movement of an anatomical object. Each image sequence is supplied to a classifier module that employs one or more machine learning algorithms to derive at least one score for each image sequence that indicates a predicted success or quality of the result of the image processing task when applied to the given image series. This allows to assess in advance which one of the plurality of image series is most likely to result in the best (e.g. highest quality or largest amount of information) result by the image processing task. This allows to maximize the quality of the image processing result without having to actually process each image series with the image processing task, which would consume a large amount of processing resources and consume time.

Description

Technical Field

[0001] This invention relates to methods in the field of medical image processing. Background Technology

[0002] In the field of medical imaging, one or more automated image analysis routines are frequently used to process acquired patient image sequences to derive clinically relevant information. This can include additional anatomical information, additional graphical details, or other clinically relevant information such as measurements. A common image processing task is image segmentation. Image processing tasks can be applied to images from any modality, such as ultrasound, MRI, CT, X-ray, PET, or any other.

[0003] The success of an image processing procedure, such as the accuracy or quality of the results, depends on the quality of the image sequences used. Often, multiple image sequences are acquired and the processing is applied to each sequence to derive a different set of results. Clinicians can then select the result with the highest quality, containing the most information, or otherwise being the most successful. This avoids situations where poor image processing results necessitate recalling the patient for a report scan to improve the outcome.

[0004] Repeated image processing tasks are time-consuming and resource-intensive. Improvements to avoid poor image processing results would be beneficial.

[0005] Reference paper: Frouin F et al: "Factor analysis of the left ventricle byechocardiography (FALVE): a new tool for detecting regional wall motionabnormalities", European Journal of Echocardiography, Harcourt Publishers, Edinburgh, GB, vol. 5, no. 5, 1 October 2004 (2004-10-01), pages 335-346. This paper describes a method for tracking cardiac wall motion using factor analysis for ultrasound image sequences.

[0006] Other reference paper: L Yu Juan et al: "30 Ultrasound Spine Image Selection Using Convolution Learning-to-Rank Algorithm", 2019 41st Annual International Conference Of The IEEE Engineering In Medicine And Biology Society (EMBC), IEEE, 23 July 2019 (2019-07-23), pages 4799-4802. This paper describes a method for selecting the best quality images for spine carving using artificial neural networks. Summary of the Invention

[0007] This invention is defined by the claims.

[0008] According to an example of an aspect of the invention, a system is provided for evaluating candidate image series of periodically moving anatomical objects for application to an image processing task. The system includes a classifier module comprising at least one machine learning algorithm adapted (i.e., trained) to receive image series of periodically moving anatomical objects as input and generate at least one score as output, the at least one score representing a predictive metric of success for a particular image processing task (if applied to the image series). The system further includes a control module adapted to: receive multiple image series of periodically moving objects; supply each image series as input to the classifier module to obtain at least one score for each of the multiple image series; and identify an optimal subset of the multiple image series based on the at least one score for each image series for application to the image processing task.

[0009] Compared to multiple received image series, the optimal subset identified contains fewer image series. In some cases, it can correspond to only a single image series.

[0010] The embodiment is based on an application selection algorithm that pre-evaluates which image series among several different candidate image series are most likely to produce the best results when a specific image processing task is applied to it. This is based on analyzing or processing the image series itself, without applying the image processing task. Therefore, this saves time and processing resources. The analysis of candidate image series is performed automatically using a classifier incorporating one or more artificial intelligence algorithms. It reflects knowledge gained through a training process, during which one or more algorithms are trained to predict likely high-quality results for the image processing task.

[0011] The optimal subset of the multiple image series means the subset that has the highest probability of success or the highest overall prediction scale for the image processing task based on at least one rating.

[0012] Each image series can include multiple images, each corresponding to a specific point in time. Each image series can therefore correspond to a temporally ordered sequence of images. This can span the entirety or a portion of one or more movement cycles of the anatomical object.

[0013] The anatomical object is a periodically moving anatomical object. A movement cycle can be understood as containing multiple phases. Each candidate image series can cover at least one movement cycle and preferably multiple cycles.

[0014] The image processing task can be an anatomy-specific task, meaning an image processing task suitable for deriving information from one or more images of a specific anatomical body. It can also be a clinical image processing task, meaning an image processing task suitable for deriving clinically relevant information from a series of input images. Clinically relevant information can mean information related to the size of one or more anatomical features, one or more physiological parameters, the clinical state of one or more anatomical features (e.g., normal, diseased, etc.), or any other relevant information.

[0015] The method may also include generating a data output that indicates an optimal subset of multiple image series for identification. This can be provided, for example, to a further image processing module. In some examples, the data output can be transmitted to a user interface for presentation to a user via sensor output, such as for display on a display device.

[0016] The system may further include an image processing module adapted to receive a series of images as input, apply image processing tasks to the input series of images, and generate an output series of processed images. The control module may be adapted to supply the image processing module with an optimal subset of the identified image series. The control module may also be adapted to supply the image processing module only with the optimal subset of the identified image series, and not with the remaining portions of the image series not included in the identified subset. This avoids wasting processing resources applying image processing operations to image series predicted to result in low-quality image processing results.

[0017] In some cases, the output of an image processing task includes one or more processed images. It may include one processed image from each image in a series of candidate images. In other examples, the output of an image processing task may include another type of information or data derived by processing the image, such as a list of anatomical features identified in the image series, dimensional measurements of anatomical features within the image series, or temporal measurements of anatomical processing within the image series. There may be one output per image in the image series, or a smaller number of outputs for the entire image series.

[0018] The control module may be adapted to store records of multiple received image series in a memory (e.g., local memory or cache), and may be adapted to remove, delete or discard each received image series that is not included in the best subset of the identified image series from the memory, and not supply these image series to the image processing module.

[0019] In some examples, this step can be performed before supplying the optimal subset of the image series to the image processing module. This maximizes memory capacity, as unwanted image series are discarded at the earliest possible time.

[0020] One or more machine learning algorithms in the classifier module can be adapted to generate multiple distinct scores, each representing a different metric of predictive success for the image processing task. Each distinct score can be associated with a different attribute or quality of a given image series, which is known to be related to the quality of the image processing task applied to the output of that image series (i.e., known to be successful or of high quality in predicting the output of the image processing task). By basing the selection of the optimal image series on multiple distinct scores, each mapped to a different analyzable attribute of the image series, this improves the robustness and reliability of the evaluation because multiple independent factors are considered.

[0021] The control module may also be adapted to: determine an overall score for each of the multiple image series based on the corresponding multiple scores for each image series; and identify the optimal image series for applying the image processing task based on the overall score for each image series.

[0022] The overall score can be determined using predefined evaluation functions or operations. This can include one or more weighted averages of multiple scores.

[0023] The method may include sorting the plurality of image series based on at least one rating or an overall rating. The control module may be configured to generate data output indicating the sorting of the image series for transmission to a user interface device. The sorting may be displayed on the display device of the user interface. The user interface may be adapted to allow a user to select any one of the plurality of candidate image series. The displayed sorting assists the user in making their selection. In some examples, the selection may be transmitted to an image processing module, and the image processing module may be adapted to apply the image processing task to the image series selected by the user.

[0024] The at least one machine learning algorithm may be a machine learning algorithm trained using a training dataset, which includes multiple series of sample images, each of which is manually labeled with the at least one rating.

[0025] In one set of embodiments, the image processing task includes image segmentation of one or more anatomical regions of the periodically moving anatomical object.

[0026] In this set of embodiments, in some examples, the at least one rating may include a rating indicating a predetermined visual (i.e., graphic, spatial, shape) correspondence between a shape or contour generated in the segment when applied to the image series and a shape or contour existing in the images of the image series. This may be referred to as a visual confidence score. It relates to the degree of correspondence between the source image series and the graphic information in the segment.

[0027] Alternatively or additionally, the at least one rating may include an indication of the predicted correspondence between the mesh geometry generated in the segment if applied to the image series and the geometry of a predefined anatomical object of interest. This may be referred to as a mesh rationality rating.

[0028] The known geometry can be a typical geometry of the anatomical object.

[0029] According to one or more embodiments, the at least one score may include a score indicating the predictive consistency or correspondence between the following two: The result of an image processing task targeting at least one image from an image series occurring during the first phase of the movement cycle of the anatomical object; and The result of an image processing task for at least one other image in the image series that occurs during other phases of the movement cycle of the anatomical object.

[0030] This can be called segment consistency scoring.

[0031] In at least one set of embodiments, the image series is an ultrasound image series, meaning that each image series includes a series of ultrasound images.

[0032] In some embodiments, the system for which protection is sought may further include an ultrasound imaging device, wherein the control module is adapted to receive the plurality of image series from the ultrasound imaging device.

[0033] According to an example of another aspect of the invention, a computer-implemented method is provided, comprising: receiving a plurality of image series of periodically moving anatomical objects; applying a classifier operation to each of the plurality of image series, wherein the classifier operation includes at least one machine learning algorithm adapted (i.e. trained) to receive the image series of periodically moving objects as input and generate at least one score as output, the at least one score representing a predictive metric of success for a particular image processing task to be applied to the image series; and identifying an optimal subset of the plurality of image series for applying the image processing task based on the at least one score for each image series.

[0034] In some embodiments, the method further includes applying the image processing task to a best subset of the identified plurality of image series. The method preferably includes applying the image processing task only to the identified best subset and not applying the image processing task to any image series among the received plurality of image series that are not included in the best subset of the identified image series.

[0035] According to other aspects of the present invention, a computer program product including computer program code is provided, the computer program code being configured to, when run on a processor, cause the processor to perform a method according to any of the examples or embodiments summarized above or described below, or to perform a method according to any claim of the present application.

[0036] These and other aspects of the invention will become apparent and clear with reference to the embodiments described below. Attached Figure Description

[0037] To better understand the invention and to more clearly illustrate how the invention can be implemented, reference will now be made only by way of example to the accompanying drawings, in which: Figure 1 An example system according to one or more embodiments is schematically depicted; and Figure 2 The steps of an example method according to one or more embodiments are summarized. Detailed Implementation

[0038] The invention will be described with reference to the accompanying drawings.

[0039] It should be understood that the detailed descriptions and specific examples, while illustratively indicating exemplary embodiments of the apparatuses, systems, and methods, are intended for illustrative purposes only and are not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatuses, systems, and methods of the present invention will become better understood from the following description, the appended claims, and the accompanying drawings. It should be understood that the drawings are schematic only and not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to indicate the same or similar parts.

[0040] This invention provides a method for achieving more accurate results when applying image processing tasks to a patient's medical image series without significantly increasing processing resources. The proposed system and method are based on the following: receiving multiple image sequences of a specific anatomical region, each image sequence capturing the periodic movement of the anatomical object. Each image sequence is applied to a classifier module, which employs one or more machine learning algorithms to derive at least one score for each image series, indicating the predictive success or quality of the image processing task's outcome if applied to a given image series. This allows for advance evaluation of which of the multiple image series are most likely to produce optimal (e.g., highest quality or maximum information content) results from the image processing task. This allows for maximizing the quality of the image processing results without actually processing each image series with an image processing task, which would consume significant processing resources and time. The classifier model can achieve predictions much faster.

[0041] Image processing tasks can be anatomically specific, meaning they are suitable for deriving information from one or more images of a specific anatomy. They can also be clinical image processing tasks, meaning they are suitable for deriving clinically relevant information from an input series of images. Clinically relevant information can mean information related to the size of one or more anatomical features, one or more physiological parameters, the state of one or more anatomical features (e.g., normal, diseased, etc.), or any other.

[0042] In medical imaging, such as ultrasound imaging, numerous image sequences are often acquired for each patient case. Using inappropriate image sequences can affect the derived clinical measurements or diagnoses. Therefore, given a set of image sequences, clinicians are expected to perform this task using the sequences that contain the most relevant information for a given clinical task.

[0043] If users have to manually perform the selection by looping through all available sequences, this would waste valuable time. Furthermore, it doesn't guarantee the accuracy of the user's selection of the optimal image series.

[0044] Choosing the optimal image sequence is non-trivial and depends on various criteria, often related to the subsequent image processing task to be applied to the image series. While image quality is a factor, a score based solely on image quality does not necessarily provide an accurate prediction of the success of the image processing task. By way of example, some image sequences may be unsuitable for a particular clinical task because they cover the wrong anatomical field of view (FOV) (incomplete coverage of the anatomy) or they do not cover the required time window (e.g., insufficient temporal coverage of the cardiac cycle to obtain a view of the target cardiac phase).

[0045] Two image sequences may exhibit similar quality, but it may be unclear which sequence will lead to the optimal result for the designed clinical image processing task. For example, this could be the case where the target anatomical region (e.g., the entire left ventricle of the heart) is not clearly visible in either image sequence. In such cases, it is unclear how to distinguish between the different image series and identify which series will lead to the best image processing result. Therefore, the optimal sequence is not necessarily the one with the highest anatomical coverage or the one with the highest image quality.

[0046] Therefore, a method capable of automatically determining which of multiple image sequences is best suited for a specific image processing task would be valuable. Preferably, the selection method is automatic. Preferably, it is operable to be executed in real time with image acquisition.

[0047] Figure 1 An example system 10 according to one or more embodiments of the present invention is illustrated. This system is used to evaluate a candidate image series or sequence of periodically moving anatomical objects in relation to applying an image processing task to a candidate image series.

[0048] The system includes a control module 14. The control module is adapted to receive multiple candidate image series 12 or sequences of periodically moving anatomical objects.

[0049] The system also includes a classifier module 16 operatively coupled to the control module. The classifier module includes or consists of at least one machine learning algorithm adapted (i.e. trained) to receive a series of images of periodically moving objects as input and generate at least one score as output, which represents a predictive measure of success or accuracy for a particular image processing task (if the series of images is to be applied).

[0050] The control module 14 is also adapted to supply each received image series 12 as input to the classifier module 16, so as to obtain at least one score for each of the plurality of image series. The control module is also adapted to identify an optimal subset of the plurality of image series for applying the image processing task based on the at least one score for each image series.

[0051] The best subset of multiple image series, for example, means the subset of the overall success measure of the image processing task with the highest probability of success or the highest predictive success, based on at least one rating.

[0052] Embodiments of the present invention are applicable to a variety of different image processing tasks. Typically, these can be clinical image processing tasks, meaning image processing tasks suitable for deriving or extracting clinically relevant information from a series of input images. This can be exemplified by information related to the contours or locations of one or more anatomical features, the dimensions of one or more anatomical features, one or more physiological parameters, the clinical state (e.g., normal, diseased, etc.) of one or more anatomical features, or any other clinically relevant information.

[0053] According to some examples, the system may optionally further include an image processing module 18, operatively coupled to the control module 14 and adapted to receive an image series as input, apply relevant image processing tasks to the input image series, and generate an output processed image series. The output processed image series may, for example, include a processed version of each image in the input image series (e.g., a segmented version of each image). Optionally, it may include clinically relevant information extracted from the image series.

[0054] At least one rating may be related to a specific analyzable property or quality of the image series that is known to be associated with the quality of the output of the image processing task when applied to a given image series (i.e., known to be successful or of predictable quality for the output of the image processing task).

[0055] The optimal subset of multiple image series can include a single image series or multiple image series.

[0056] The control module 14 can be adapted to supply the image processing module with only the best subset of the image series for recognition, and not supply the rest of the received image series.

[0057] The control module 14 may include local memory or a cache. The control module may be adapted to store records of multiple received image series in local memory, and to remove, delete, or discard at least each of the received image series that is not included in the best subset of the identified image series from memory, without supplying image series to the image processing module.

[0058] It should be noted that, although in Figure 1In the example above, the classifier module 16, control module 14, and optional image processing module 18 are shown as separate components within system 10, but this is not the basic model. Their related functions can be distributed among one or more components in different ways. For example, the functions of different modules can be integrated and executed by a single element, such as a single controller or processor, or their functions can be otherwise distributed among one or more elements or components. Optionally, the function of each module can be executed by a separate controller or processor within system 10.

[0059] Moreover, despite Figure 1 The control module 14 is shown as an integrated control unit, but this is for illustrative purposes only. In this embodiment, and in any embodiment described throughout this disclosure, the functions performed by the controller may be performed by a distributed collection of more components, such as multiple control elements or processors, or by processing elements included in one or more other components within the device.

[0060] Each image series may include multiple images, each corresponding to a specific time point. Each image series may therefore correspond to a temporally ordered sequence of images. This may span the entirety or a portion of one or more movement cycles of the anatomical object. In some examples, the classifier module may be adapted to derive at least one score based on all images forming the image series or only selected images forming the image series. For example, it is known that the relevant anatomical features of interest are best interpreted at a specific stage of the movement cycle. Therefore, in some examples, the classifier module may be adapted to extract one or more images from each received image series corresponding to a predetermined stage of the anatomical object's movement cycle, and derive at least one score based solely on the selection of said extracted images.

[0061] In some examples, one or more of the received image series may include images spanning multiple movement cycles of the anatomical object. In these cases, in some examples, the classifier module may be adapted to extract an image corresponding to only one selected movement cycle from the multiple movement cycles.

[0062] The embodiments utilize one or more machine learning algorithms to derive at least one score for each image series.

[0063] A machine learning algorithm is any self-trained algorithm that processes input data to produce or predict output data. Here, the input data includes a sequence or series of medical images and the output data includes at least one score (if to be applied to a series of images) that indicates the success or accuracy of prediction for a particular image processing task.

[0064] Suitable machine learning algorithms for use in this invention will be apparent to those skilled in the art. Examples of suitable machine learning algorithms include decision tree algorithms and artificial neural networks. Other machine learning algorithms, such as logistic regression, support vector machines, or Naive Bayes models, are suitable alternatives.

[0065] The structure of an artificial neural network (or simply a neural network) is inspired by the human brain. A neural network consists of layers, each containing multiple neurons. Each neuron performs a mathematical operation. Specifically, each neuron can include different weighted combinations of a single type of transformation (e.g., transformations, activation functions, etc. of the same type, but with different weights). In processing input data, the mathematical operation of each neuron is performed on the input data to produce a digital output, and the output of each layer in the neural network is sequentially fed to the next layer. The final layer provides the output.

[0066] Methods for training machine learning algorithms are known. Typically, such methods involve obtaining a training dataset, including training input data entries and corresponding training output data entries. An initialized machine learning algorithm is applied to each input data entry to generate a predicted output data entry. The error between the predicted output data entry and the corresponding training output data entry is used to modify the machine learning algorithm. This process can be repeated until the errors converge and the predicted output data entries are sufficiently similar to the training output data entries (e.g., ±1%). This is generally known as a supervised learning technique.

[0067] For example, in machine learning algorithms formed by neural networks, the mathematical operations (weighting) of each neuron can be modified until the errors converge. Known methods for modifying neural networks include gradient descent, backpropagation, and so on.

[0068] The training input data entries correspond to a series of example images. The training output data entries correspond to at least one score for each series of images.

[0069] One or more machine learning algorithms in the classifier module can be adapted to generate multiple distinct scores, each representing a different measure of predictive success for the image processing task. Each distinct score can be associated with a different attribute or quality of a given image series, known to be related to the quality of the output of the image processing task when applied to that image series (i.e., known to be the predictive success or quality of the image processing task output). By basing the selection of the optimal image series on multiple distinct scores, each mapped to a different analyzable attribute of the image series, this improves the robustness and reliability of the evaluation due to the consideration of multiple independent factors. In some examples, each score can be determined by a different machine learning algorithm from the multiple machine learning algorithms included in the classifier module. Each score can be specifically trained to determine a single, particular score.

[0070] The control module may also be adapted to: determine an overall score for each of the multiple image series based on the corresponding multiple scores for each image series; and identify the optimal image series for the applied image processing task based on the overall score for each image series.

[0071] The overall score can be determined using predefined evaluation functions or operations. This can include one or more weighted averages of multiple scores. The weights can be determined in part based on the values ​​of the different scores and / or based on thresholds for different scores. If a score is, for example, below or above a predefined threshold, the weight can be set to zero.

[0072] In some examples, the image sequence with the highest overall score can then be used for further processing, for example, by being supplied to image processing module 18. The optimal subset of the image series can be configured to contain a predefined number of image series.

[0073] The classifier module can be adapted to generate scores for a single image processing task, or to generate scores for any one of multiple different image processing tasks. For example, it can be operated such that each of several different modes is configured to generate at least one score for a series of input images in relation to a different corresponding image processing task. Thus, each image processing task can have a subset of its own score set.

[0074] In some embodiments, the stack of received candidate image sequences can be sorted and / or categorized based on an overall score for each image series. The sorted or categorized set of image sequences can be provided to a user on a user interface device, such as a display device. The user interface can allow the user to select a desired subset of one or more image series for the applied image processing task.

[0075] One image processing task particularly suitable for embodiments of the present invention is image segmentation. Specifically, the image processing task may include image segmentation of one or more anatomical regions of a periodically moving anatomical object. This may include identifying the contours of one or more anatomical features or regions. It may include generating a mesh representing the contours or boundaries of a particular anatomical object or feature.

[0076] A particular example embodiment of the invention will now be summarized in more detail. This example is designed to identify the optimal subset of multiple received image series for applying an image segmentation process.

[0077] In this example, at least one rating may include a rating indicating the predicted correspondence between the shape or contour generated in the segmentation and the shape or contour existing in the images of the image series. This can be referred to as a visual confidence score. A visual confidence score effectively represents the degree to which the segmentation result is supported by evidence from the images. Segmentation algorithms may employ prior anatomical models, and thus in some cases the generated segments may be largely derived from shape interpolation, with only limited image support. In this latter case, most segments are based solely on prior heart shape information. The purpose of the visual confidence score is to assess the degree of visual or image correspondence between the shape, contour, or geometry of the segment and the shape, contour, or geometry of the underlying image on which it is based.

[0078] Visual confidence scores can be derived using machine learning algorithms for a given series of images. The machine learning algorithm can be trained using a set of training data including example image series, each of which has been manually labeled with a visual correspondence score. The visual correspondence score can be derived based on applying an relevant image segmentation process to the image series in advance, after which the user manually evaluates the visual correspondence between the segments and the shape and contour of the image series. Applying the segmentation process can include applying the segmentation process to individual images in the image series to derive a segment for each image. The visual correspondence score for an image series can be based on the visual correspondence scores of the entire set of images forming the series, such as the average score.

[0079] Once the ratings have been manually derived and the training data has been labeled, the training image data can optionally be analyzed to identify image features associated with the ratings (e.g., the local distribution of target point distances). The machine learning algorithm can be configured to specifically analyze these features when generating predicted ratings such as image series.

[0080] Additionally or optionally, at least one rating may include a rating indicating the predicted correspondence between the mesh geometry generated in the segment and the predefined anatomical object of interest. This rating may be referred to as a mesh rationality rating.

[0081] This rating effectively indicates the degree to which the resulting segmentation will be anatomically plausible. For example, if the anatomical object of interest (i.e., the anatomical object being segmented) is the whole or a part of the heart, prior knowledge of the geometry of the whole or relevant parts of the heart can be used to assess the extent to which the generated mesh is geometrically applied to the expected geometry.

[0082] Grid rationality scores can be derived using machine learning algorithms for a given series of images. The machine learning algorithm can be trained using a training dataset comprising multiple example image series, each manually labeled with a grid rationality score. The grid rationality score is derived in advance by applying a relevant image segmentation process to each image series. The user then manually evaluates the correspondence between the segmented grid and the known or expected geometry for the relevant anatomical object based on clinical knowledge of the expected anatomical shape.

[0083] At least one rating may additionally or optionally include the predictive consistency or correspondence between: the result of image segmentation for at least one image from a series of images transmitted during a first phase of the movement cycle of the anatomical object; and the result of image segmentation for at least one other image from a series of images occurring during other phases of the movement cycle of the anatomical object. The first phase may be a phase at any point in the movement cycle (i.e., it does not need to be the chronological start phase of the cycle).

[0084] This rating can be referred to as segmentation consistency. The segmentation process can be used to segment the heart, as illustrated by the example. Segmentation consistency can be correlated with the consistency between segmentation results derived separately at the end-diastolic (ED) and end-systolic (ES) stages. This can produce a rating for each image sequence.

[0085] Each of the scores above can be represented by a numeric value within a predefined range of possible values. One or more scores can instead take the form of binary values, such as true / false or pass / fail. For example, binary values ​​can be used for segmented consistency scoring, and optionally, numeric values ​​can be used for other scores.

[0086] In some examples, the classifier module can be adapted to generate at least two of the three ratings above. In some examples, all three ratings above can be generated by the classifier module.

[0087] For example, in operation, multiple input image series 12 are received. Each is supplied as input to classifier module 16, generating two or more scores for each image series.

[0088] After each image series is individually scored using two or more different ratings, a predetermined function is used to obtain an overall score for each of the multiple input image series. By way of a non-limiting example, the overall score can be calculated based on a weighted average of the scores. However, other examples of the predetermined function for combining multiple scores into a single overall score can also be considered. This can be, for example, a linear or non-linear function. In some examples, it can utilize an algorithm with one or more steps for filtering, processing, or otherwise manipulating the scores to derive the overall score. The predetermined function for combining multiple scores into a single overall score can be recorded in local data storage, and the control unit can retrieve the predetermined function during operation.

[0089] In some examples, in addition to calculating the overall score, a threshold range can be applied to at least one score, and if at least one of the scores falls outside the threshold range, the candidate image series is excluded from the subset of the best image series. The overall score can then be calculated for each image series whose score falls within the threshold range.

[0090] Based on the overall score, a subset of one or more image series is selected for the applied image processing task.

[0091] By way of illustration, a function used to generate the overall score can be of the following form. Image sequences with [segment consistency score] = [false] and [mesh reasonableness] for any image (frame) > [X] are excluded, where X is a predefined value. The overall score is then calculated as a weighted average of the visual confidence score and the mesh reasonableness score of the processed frames. If the anatomical object of interest is the heart, in some examples, only ED and ES frames of each series may be processed. Optionally, each image sequence that is not excluded is sorted according to its corresponding overall score.

[0092] One possible application of this invention is for representing sequences of ultrasound images of the heart. These can be captured, for example, using transesophageal echocardiography (TEE) or transthoracic echocardiography (TTE) probes. Image processing tasks can be segmentation of the entire heart or a portion thereof (e.g., the left ventricle). Segmentation can be used in practice for various clinical functions, including quantification of the physical dimensions of cardiac cross-sections, quantification of hemodynamic parameters, interventional planning, and visualization. Segmentation can be model-based.

[0093] In some examples, the classifier module may be adapted to determine at least one score based on the entire set of images forming each image series (e.g., image-by-image), or only based on selected one or more images from the images forming each image sequence (e.g., only those images corresponding to a specific phase of the cardiac cycle (e.g., ED and ES image frames)). In some examples, for each sequence comprising more than one cardiac cycle, the best cardiac cycle from that image sequence may be identified (e.g., using one or more machine learning algorithms) and compared with the best cardiac cycles from other sequences from the received plurality of image sequences.

[0094] By way of example, selecting a single TEE image series from multiple image series may involve first extracting ED and ES stage images from each of the multiple image series. The three scores listed above can then be evaluated for the ED stage images for each TEE image sequence. For each ED image, the segment consistency score referred to above can be evaluated by comparing the ED image with the corresponding ES image extracted from the same image series. If the segment consistency score is below a defined minimum threshold, that value can be taken to indicate failure, in which case the corresponding image series is discarded. The optimal image series is selected based on the scores of the ED images for each image series.

[0095] In other embodiments, each received image series may not represent a temporally ordered sequence of images, but rather a set of 2D slices passing through a 3D anatomical region. The same general principle referred to above for selecting the optimal subset of an image series can also be applied in this case.

[0096] According to one or more embodiments, system 10 may further include an ultrasound imaging device, wherein control module 14 is adapted to receive a plurality of image series from the ultrasound imaging device. The ultrasound imaging device may include an ultrasound transducer unit, such as an ultrasound probe, for acquiring ultrasound data. The imaging device may also include components for deriving ultrasound images based on the acquired ultrasound data.

[0097] According to examples of other aspects of the present invention, a computer-implemented method is provided. Figure 2 The steps of an example method 40 according to one or more embodiments are summarized herein.

[0098] The method includes receiving multiple image series of 42 periodically moving anatomical objects. The method also includes applying a classifier operation of 44 to each of the multiple image series, wherein the classifier operation includes at least one machine learning algorithm. The at least one machine learning algorithm is adapted to receive the image series of periodically moving objects as input and generate at least one score as output, the at least one score representing a predictive metric of success for a particular image processing task (if to be applied to the image series). The method also includes identifying an optimal subset 50 of 46 of the multiple image series for applying the image processing task based on the at least one score for each image series. Optionally, the remainder 52 of the received multiple image series can be discarded. Optionally, the method may further include applying an image processing task 54 to the optimal subset 50 of the identified multiple image series. Preferably, the image processing task is applied only to the optimal subset 54 of the identified multiple image series.

[0099] According to other aspects of the present invention, a computer program product including computer program code is provided, the computer program code being configured to, when run on a processor, cause the processor to perform a method according to any example or embodiment summarized above or described below or according to the claims of this application.

[0100] Those skilled in the art, when practicing the claimed invention, can understand and implement various modifications to the disclosed embodiments through study of the accompanying drawings, this disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the non-limiting words "a" or "an" do not exclude a plurality.

[0101] A single processor or other unit can perform the functions of several items cited in the claims.

[0102] The only fact is that certain means cited in common but different claims do not indicate that combinations of these means cannot be used.

[0103] Computer programs can be stored or distributed on suitable media, such as optical storage media or solid-state media that are used with or as part of other hardware, but they can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0104] If the term “suitable” is used in the claims or description, note that the term “suitable” is intended to be equivalent to the term “configured as”.

[0105] No reference numerals in the claims should be construed as limiting the scope.

Claims

1. A system (10) for evaluating a series of images (12) of an anatomical object undergoing periodic movement to apply an image processing task, the system comprising: The classifier module (16) includes at least one machine learning algorithm adapted to receive a series of images of periodically moving objects as input and generate at least one score as output, the score representing a predictive measure of the success of a particular image processing task if the series of images is to be applied. as well as Control module (14), which is suitable for: Receive (42) multiple image series of the periodically moving anatomical object, Each image series is supplied (44) as input to the classifier module to obtain at least one score for each of the plurality of image series, and (46) Identify (50) the optimal subset of the plurality of image series for applying the image processing task based on the at least one score for each image series.

2. The system (10) according to claim 1, wherein, The system further includes an image processing module (18) adapted to receive an image series (12) as input, apply the image processing task (54) to the input image series, and generate an output processed image series.

3. The system (10) according to claim 2, wherein, The control module (14) is adapted to supply the image processing module (18) with only the best subset (50) of the identified image series.

4. The system (10) according to claim 3, wherein, The control module (14) is adapted to store records of the received multiple image series (12) in a memory, and is adapted to delete from the memory at least each image series that is not included in the best subset (50) of the identified image series, without supplying the image series to the image processing module (18).

5. The system (10) according to any one of claims 1-4, wherein, The one or more machine learning algorithms of the classifier module (16) are adapted to generate multiple different scores, each score representing a different measure of the prediction success of the image processing task.

6. The system (10) according to claim 5, wherein, The control module (14) is also adapted to: An overall score for each of the multiple image series is determined based on the corresponding multiple scores for each image series; and The optimal subset of the plurality of image series for applying the image processing task is identified based on the overall score for each image series.

7. The system (10) according to claim 6, wherein, Identifying the best subset of the plurality of image series includes sorting the plurality of image series (12) according to the at least one score or the overall score.

8. The system (10) according to any one of claims 1-4, wherein, The at least one machine learning algorithm is a machine learning algorithm that has been trained using a training dataset comprising multiple series of sample images, each series of sample images being manually labeled with the at least one rating.

9. The system (10) according to any one of claims 1-4, wherein, The image processing task includes image segmentation of one or more anatomical regions of the periodically moving anatomical object.

10. The system (10) according to claim 9, wherein, At least one rating includes: A score indicating the predicted correspondence between the shape or contour generated in the segmentation when applied to the image series and the shape or contour existing in the image series; and / or A score indicating the predicted correspondence between the mesh geometry generated in the segmentation when applied to the image series and the geometry of the predefined anatomical object of interest.

11. The system (10) according to any one of claims 1-4, wherein, The received image series (12) includes multiple ultrasound image series.

12. The system (10) according to any one of claims 1-4 further includes an ultrasound imaging device, and wherein, The control module (14) is adapted to receive the plurality of image series (12) from the ultrasound imaging device.

13. A computer-implemented method (40), comprising: Receive (42) multiple image series of periodically moving anatomical objects; A classifier operation is applied to each of the plurality of image series, wherein the classifier operation includes at least one machine learning algorithm adapted to receive an image series of periodically moving objects as input and generate at least one score as output, the at least one score representing a predictive measure of the success of a particular image processing task if the image series is to be applied. (46) Identify (50) the optimal subset of the plurality of image series for applying the image processing task based on the at least one score for each image series.

14. The method of claim 13 further comprises applying the image processing task only to the best subset (50) of the identified plurality of image series (12).

15. A computer program product comprising computer program code configured to cause the processor to perform the method according to claim 13 or 14 when executed on a processor.

Images