Ultrasonic image acquisition

Abstract

A method and system for automatically calculating one or more optimal planes through a 3D volume in order to automatically detect one or more suspicious disease features within an input 3D ultrasound image and obtain the images that are most useful for confirming or further analyzing the suspicious disease features. The determined one or more optimal planes are used to control the acquisition by an ultrasound acquisition system of new 2D images corresponding to the determined optimal planes.

Description

Technical Field

[0001] The present invention relates to ultrasonic image acquisition.

Background Art

[0002] In conventional ultrasonic examinations, when an ultrasonic examiner performs an examination on one or more planes to be recorded, manual determination must be made. The planes are recorded for diagnosis or to prove a diagnosis performed in real time while the ultrasonic examiner views the images. The planes may also be recorded to perform measurements of specific anatomical features.

[0003] Each additional plane to be recorded is time-consuming. In the art, systems are known that automatically record specific standard planes for a standardized procedure without the need for an ultrasonic examiner to explicitly reconfigure a protocol or relocate a transducer. This is achieved, for example, by placing a target anatomical structure within a 3D field of view (FOV) by model-based segmentation and deriving slices through the segmented anatomical structure obtained using a dedicated beamforming protocol. See, for example, patent applications WO 2015 / 087218 and WO 2014 / 162232.

[0004] See also US 2014 / 039318 A1, which discloses a method for the automatic detection of suspicious abnormalities in ultrasonic breast images.

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the context of an examination for disease screening or diagnosis purposes, the acquisition of planes is guided by the judgment of the ultrasonic examiner when identifying suspicious features in the images acquired in real time.

[0006] In particular, for untrained sonographers, there is a possibility that incidental findings or suspicious areas may be overlooked. In this case, the diagnosis may be inaccurate, or in the worst-case scenario, serious situations may be missed. Each additional acquisition takes time and lengthens the examination time, so the sonographer must balance a short examination time with the possibility of missing important findings, while at the same time always acquiring the maximum number of views to ensure that all possible findings are covered. In most practical clinical situations, sonographers tend to prefer shorter examination times due to lack of ability and the low probability of incidental findings occurring. However, this leaves the risk of missing potentially important disease indicators.

Means for Solving the Problem

[0007] This invention is defined by the claims.

[0008] According to an embodiment according to one aspect of the present invention, A computer-implemented method, Receiving, as input from an ultrasound acquisition system, a 3D ultrasound image dataset having at least one 3D image frame across a 3D field; Processing the input 3D image frame and applying a disease detection module adapted to detect one or more suspicious disease features from a predefined set of possible disease features; For each 3D image frame, applying a planar mapping module adapted to determine a set of one or more 2D slices through the 3D field of the image frame based on the output from the disease detection module to image one or more disease regions associated with the detected one or more disease features; Generating control instructions for output to an ultrasound acquisition system to effect automatic acquisition of a set of one or more 2D slices, for example using B-mode imaging, for example using 2D imaging A method is provided that has

[0009] Accordingly, embodiments of the present invention are based on the concept of using a disease detection module to automatically detect suspicious disease features within an acquired 3D image and, in addition, automatically determining optimal image acquisition instructions for acquiring a new 2D image frame that captures one or more planes that provide the best representation of the anatomical structure for analyzing the disease features for diagnostic purposes. By controlling the acquisition of the new 2D image, it is possible to acquire a higher resolution 2D frame of the relevant region, rather than simply extracting slices from the already acquired 3D image, or to acquire frames having those acquisition parameters / characteristics optimized for subsequent diagnosis. The acquired 3D image stream may be of lower quality or resolution, which is suitable for investigating the anatomical region but may not be of sufficient quality to perform the final diagnostic analysis. Furthermore, by automatically determining the optimal 2D image plane to be captured, the user does not need to make this determination and the risk of potentially overlooked findings is reduced.

[0010] In some embodiments, the input 3D ultrasound image dataset includes a stream of 3D image frames and the steps of the method are performed in real time with the reception of each 3D image frame.

[0011] In some embodiments, a plane mapping module is adapted to use a look-up table to select a predetermined plane based on one or more detected disease features. The plane may be, for example, a plane predetermined as being most suitable for the clinical evaluation of a given detected anatomical feature.

[0012] In some embodiments, a disease detection module is adapted to generate a 3D spatial map of a disease region within the 3D field of an image frame corresponding to a disease feature for at least one 3D image frame.

[0013] In some embodiments, the planar mapping module is adapted to receive, as input, a 3D map of a detected disease region within a 3D field and, depending on the map, determine a set of one or more 2D slices through the 3D field that intersect the disease region.

[0014] In some embodiments, the planar mapping module is adapted to perform a spatial fitting of a plane to one or more disease regions and determine a set of one or more 2D slices that intersect all of the regions and optionally satisfy one or more additional constraints.

[0015] Referring to the one or more additional constraints, these may depend in part on the particular disease characteristics to which the disease regions correspond.

[0016] The one or more additional constraints may comprise requirements for minimizing the number of 2D slices while still intersecting all of the regions.

[0017] The one or more additional constraints may comprise one or more tolerances for the 2D slice surface orientation with respect to one or more directions.

[0018] For at least one subset of a predefined set of disease characteristics in some embodiments, the detection of disease characteristics by the disease detection module includes segmenting and classifying one or more spatial regions as suspect regions.

[0019] When the disease detection module is adapted to generate a 3D spatial map, the 3D spatial map output by the disease detection module may include a map of the segmented suspect regions.

[0020] In some embodiments, for at least one subset of a predefined set of disease features, the detection of each disease feature by the disease detection module includes calculating a 3D saliency map over a 3D field associated with the disease feature and deriving a separate classification of the 3D image associated with the feature based on the saliency map. When the disease detection module is adapted to generate a 3D spatial map, the 3D spatial map output by the disease detection module can be a saliency map. In other words, the saliency map is used as a 3D spatial map of the disease region. For example, a machine learning algorithm trained to generate one or more disease feature classifications in relation to an input image frame can generate a 3D saliency map indicating the saliency of different image positions for the final classification. In some embodiments, the planar mapping module is adapted to determine a set of one or more 2D slices based on a fitting plane of maximum saliency through the saliency map. The plane of maximum saliency means, for example, the plane that plots the path of the plane of maximum saliency, i.e., the plane containing the maximum aggregated saliency.

[0021] In some embodiments, a new 2D image may be generated in the background of an ongoing image acquisition procedure being performed by a user using an ultrasonic acquisition device. For example, in some embodiments, control instructions are adapted to cause the ultrasonic acquisition system to interleave the acquisition of a set of one or more 2D slices with any other acquisition sequence that the ultrasonic acquisition system is currently performing.

[0022] In some embodiments, the disease detection module includes at least one trained machine learning algorithm, preferably a convolutional neural network (CNN).

[0023] As briefly described above, in some embodiments, control instructions are adapted to control the acquisition of 2D slices having a higher spatial resolution than the input 3D ultrasonic image data.

[0024] In some embodiments, the method further includes receiving a set of acquired 2D slices and controlling a user interface to generate a visual output representing it.

[0025] In some embodiments, after determining a set of 2D slices, the method further includes controlling a user interface to generate a user-recognizable prompt that requests approval to acquire the set of 2D slices, and the generation of the control instructions is executed only in response to receiving, from the user interface, a user input indicating approval.

[0026] The present invention can be implemented in software form. Accordingly, another aspect of the present invention is a computer program product comprising computer program code or other code means configured to cause a processor, when executed on the processor (e.g., operably coupled to an ultrasonic acquisition system), to execute a method according to an example or embodiment of the present invention described in this document or according to any claim of this patent application.

[0027] The present invention can also be implemented in hardware.

[0028] Accordingly, another aspect of the present invention is a processing device comprising an input / output unit and one or more processors. The one or more processors, as an input from an ultrasonic acquisition system, receive a 3D ultrasonic image dataset comprising at least one 3D image frame across a 3D field, process the 3D image frame, and apply a disease detection module adapted to detect one or more suspected disease features from a predefined set of possible disease features, and based on the output from the disease detection module, apply a planar mapping module adapted to determine a set of one or more 2D slices through the 3D field of each image and image one or more disease regions associated with the detected one or more disease features, and generate control instructions for output via the input / output unit to the ultrasonic acquisition system to effect an automatic acquisition of a set of one or more 2D slices, for example using B-mode imaging.

[0029] Another aspect of the present invention is a system comprising an ultrasonic acquisition system and a processing device according to any of the embodiments described above or described in this document, the processing device being operably coupled to the ultrasonic acquisition system. The system may further comprise a user interface device in some embodiments.

[0030] These and other aspects of the present invention will be apparent from and will be elucidated with reference to the embodiments described hereinafter.

[0031] For a better understanding of the present invention and to more clearly show how it may be carried out, reference is now made, by way of example only, to the accompanying drawings.

Brief Description of the Drawings

[0032]

Figure 1

Figure 2

Figure 3

Figure 4

DETAILED DESCRIPTION OF THE INVENTION

[0033] The present invention will be described with reference to the drawings.

[0034] It should be understood that the detailed description and specific examples show exemplary embodiments of the apparatus, system, and method, but are for illustrative purposes only and are not intended to limit the scope of the present invention. These and other features, aspects, and advantages of the apparatus, system, and method of the present invention will be better understood from the following description, the appended claims, and the accompanying drawings. It should be understood that the figures are merely schematic and not drawn to scale. Also, it should be understood that the same reference numbers are used throughout the drawings to indicate the same or similar parts.

[0035] The present invention provides a method and system for automatically calculating one or more optimal planes through a 3D field in an input 3D ultrasound image to automatically detect one or more suspicious disease features and obtain the images that are most useful for confirming or further analyzing the suspicious disease features. The determined one or more optimal planes are used to control the acquisition by an ultrasound acquisition system of new 2D images corresponding to the determined optimal planes.

[0036] The new 2D images can preferably be acquired with a higher spatial resolution than the original 3D image frames, thus enabling further diagnostic analysis of the suspicious disease features using higher-quality images.

[0037] The ultimate goal is, for example, to automatically record the optimal slices for the evaluation of incidental findings or suspicious regions.

[0038] According to an advantageous embodiment, it is proposed to add to the image processing pipeline a disease detection module adapted to scan (e.g., continuously) all the acquired 3D frames for suspicious features or anomalies. From this module, optionally, a spatial map of the anatomical regions associated with these features can be derived. This map is then fed into an optimization step to calculate the optimal one or more 2D slices incorporating the mapped regions in order to confirm the findings. The latter step can be performed by a dedicated module, which is referred to in the present disclosure as a planar mapping module.

[0039] The determined optimal 2D slices can then be automatically recorded, ideally, for example, by reconfiguring the transducer arrangement of the ultrasound acquisition system, such that the ultrasound examiner is unaware, e.g., by using scan pauses or interleaved scans. The recorded slices can optionally be further processed using a verification module or a plane prediction module to check whether the acquired slices actually correspond to the target plane, i.e., whether the target slice has been hit, or whether further optimization needs to be performed until convergence.

[0040] Finally, the additionally recorded slices can be visualized to the ultrasound examiner either on demand after notification, at the end of the examination, or during a later review of the examination data.

[0041] The above represents the features of an exemplary set of embodiments and not all of the features mentioned are essential to the concept of the invention, as will become apparent in the following description.

[0042] FIG. 1 outlines the steps of an exemplary method according to one or more embodiments in a block diagram. The steps are summarized before being further described in the form of exemplary embodiments.

[0043] Method 10 includes step 12 of receiving, as input from an ultrasonic acquisition system, a 3D ultrasonic image data set including at least one 3D image frame across a 3D field. The method further includes applying a disease detection module adapted to process the input 3D image frame, and detecting one or more suspected disease features therein with a predefined set of possible disease features (14). The method further includes applying a planar mapping module adapted to determine, for each 3D image frame, a set of one or more 2D slices through the 3D field of the image frame based on the output from the disease detection module, to image one or more disease regions associated with the detected one or more disease features (16). The method further includes step 18 of generating control instructions for output to the ultrasonic acquisition system to effect automatic acquisition of a set of one or more 2D slices, for example using a 2D image mode, for example using a B-mode image.

[0044] One or more detected disease features can include physical structures or objects imaged within a 3D image frame that have one or more properties or characteristics associated with a disease state, such as a calcified wall portion, or a lesion, or a nodule. The one or more detected disease characteristics can additionally or alternatively include a classification of an entire anatomical structure (e.g., an organ), or an entire physiological or anatomical system, or a region of an anatomical structure or system, such as a particular state or characteristic associated with abnormal wall motion or abnormal shape of a particular anatomical structure. Thus, the aforementioned disease region related to a disease means a disease region that includes or overlaps at least a part of the anatomical structure or region to which a given disease feature is related or related to the diagnosis of the disease feature. If the disease feature is a physical structure or body suspected of being a disease, the disease region may naturally be or include the region containing the structure or body. If the disease feature is a more general or overall classification of a structure or system within the body, one or more disease regions can be disease regions known to be related to the diagnosis of a given detected disease feature. Knowledge of such related disease regions for a given disease feature may be implicitly learned as part of the training of a machine learning algorithm, for example, incorporated into the program of a planar mapping module, or explicitly encoded as part of one or more algorithms used by the planar mapping module.

[0045] As described above, the method can also be implemented in hardware form, for example, in the form of a processing unit configured to execute the method according to any example or embodiment described in this document, or according to any claim of this application.

[0046] To further aid understanding, FIG. 2 presents a schematic diagram of an exemplary processing device 32 configured to execute the method according to one or more embodiments of the present invention, and also schematically shows the data processing flow in more detail.

[0047] The processing device 32 includes an input / output unit 34 and one or more processors 36. The one or more processors 36 are configured to execute the following method. The method includes receiving, at the input / output unit 34, as input from an ultrasonic data acquisition system 52 (i.e., an ultrasonic scanning system), a 3D ultrasonic image data set 54 including at least one 3D image frame across a 3D field. The method further includes applying a disease detection module 42 adapted to process the input 3D image frame, and detecting one or more suspected disease features 56 therein against a predefined set of possible disease features. The method further includes applying a planar mapping module 44 adapted to determine a set of one or more 2D slices 58 through the 3D field of each image based on the output from the disease detection module 42 to image one or more disease regions associated with the detected one or more disease features. The method further includes generating control instructions 60 for output via the input / output unit 34 to the ultrasonic acquisition system 52 to effect, for example, the automatic acquisition of the determined set of one or more 2D slices, using, for example, a B-mode image and in a 2D image mode.

[0048] Another aspect of the invention is a system 30 including a processing device 32. FIG. 2 shows the processing device 32 operating within the context of such a system. The system 30 can further include an ultrasonic acquisition system 52. The system can further include a user interface device (not shown) that can be controlled in some embodiments to display a visualization of the acquired 2D image slices.

[0049] Although the disease detection module 42 and the planar mapping module 44 are illustratively shown as separate entities or components within the processing device 32, it should be noted that this is only schematic. In fact, these can be software modules that can be encoded in the program of a single processor, or their functions can be executed in a distributed manner by multiple processors. Similarly, the item labeled as one or more processors 36 that execute the method includes steps of calling modules 42 and 44, and can be a single processor or multiple processors. In fact, the functionality of the disease detection module 42 and the planar mapping module 44 can be encoded on the same processor that executes the high-level method 10 having the steps outlined above.

[0050] To clarify the general concepts summarized above, two exemplary implementations of the method according to each specific group of embodiments are described here by way of illustration. These overviews are shown in FIGS. 3 and 4. It will be understood that not all features of these specific groups of embodiments are essential to the concepts of the present invention, and they are described to assist in understanding and to provide examples for illustrating the more general concepts of the present invention.

[0051] In summary, the exemplary implementations of FIGS. 3 and 4 are similar in most respects. They mainly propose that the embodiment of FIG. 3 uses a convolutional neural network (CNN) trained for several separate classification tasks for the implementation of the disease detection module 42 and outputs a classification regarding a specific disease state, for example, the detection of wall motion abnormalities, while the embodiment of FIG. 4, instead, uses a CNN trained for one or more segmentation tasks for the implementation of the disease detection module 42 and outputs a suspicious structure indicating a potential disease (e.g., segmentation of calcification) or at least one segmentation of the body.

[0052] Both the embodiment of FIG. 3 and the embodiment of FIG. 4 begin with step 12 of receiving 3D ultrasonic image data including at least one 3D image frame across a 3D field. Optionally, the input 3D ultrasonic image data set includes a stream of 3D image frames, and the steps of the method are performed in real time with the reception of each 3D image frame.

[0053] Next, the disease detection module 42 is applied to the received 3D ultrasonic image data. As briefly described above, the disease detection module is configured to process the input 3D image frame and detect one or more suspected disease features among a predetermined set of possible disease features. To perform the detection of disease features, both the method of FIG. 3 and the method of FIG. 4 propose to utilize a trained machine learning algorithm. More specifically, both the embodiments of FIGS. 3 and 4 propose to use a trained convolutional neural network (CNN). However, it is emphasized that it is not necessary to implement the present invention to use a machine learning algorithm as a more general principle. Instead, any other type of algorithm can be utilized, for example, other types of image processing algorithms that use shape detection or model-based segmentation to detect a specific structure or to identify elements within an image that may indicate a specific disease feature.

[0054] Returning to this example, the embodiment of FIG. 3 proposes using a CNN trained to derive at least one separate classification of 3D images regarding one or more specific possible disease characteristics. As an example, the CNN can be trained to detect wall motion abnormalities. The CNN can be trained for one or several classification tasks. Thus, the output of the CNN in this case is a classification representing the detection of disease characteristics, where the disease characteristics can be related to multiple structural elements within the image region or can represent the disease state or characteristics of a generalized system or region. Other (non-limiting) exemplary classifications that the CNN can be trained to derive include irregular heartbeat, valve insufficiency, abnormal ejection fraction, diastolic dysfunction, systolic heart failure. These examples clearly relate particularly to the cardiac region, but in other embodiments, the CNN can be trained to compute classifications related to other organs, anatomical regions, or physiological systems as needed.

[0055] Optionally, when using a classification CNN, the disease detection module 42 can also use the data output from the convolutional neural network to generate a saliency map 74 representing the location identification that is the origin of the model decision. Refer to the paper "Zeiler, MD and Fergus, R. In European Conference on Computer Vision, p. 818 - 833, 2014" regarding the visualization and understanding of convolutional networks in relation to this feature implementation.

[0056] In other words, the final output of the convolutional neural network is one classification or multiple classifications. However, as part of generating the classification, the neural network can decompose the input image into localized regions and generate sub-classifications for each of these localized regions indicating their saliency with respect to the overall classification that the CNN is trained to determine. From this saliency data, a saliency map can be calculated by the CNN itself or by a disease detection module based on the information output from the CNN in combination with the classification. In other words, when determining disease features, the CNN generates saliency data for the 3D field of the input image related to the disease features, derives a separate classification of the 3D image related to the disease features based on this information, and the saliency map indicating the saliency information across the 3D field is used as a 3D spatial map of the disease region.

[0057] For purposes of comparison with the embodiment of FIG. 4, here the convolutional neural network 72 is trained to perform one or several segmentation tasks. Thus, here the detection of one or more disease features by the disease detection module 42 includes segmenting and classifying one or more spatial regions as suspect regions. As a purely illustrative example, the segmentation tasks can include for example, segmentation of a heart valve to detect, for example, leaflet defects such as flail leaflets, prolapse, or stenosis (which can be done, for example, by shape analysis), and segmentation of the left ventricular outflow tract (LVOT), for example to detect obstruction of the LVOT, and ventricular segmentation, for example to detect an enlarged heart (such as dilated cardiomyopathy) and include one or more of these.

[0058] Optionally, in the case of using the segmentation CNN 72, a 3D spatial map can be further generated and output by the disease detection module 42 including a map 75 of the segmented suspect regions.

[0059] Thus, as a more general principle, it can be seen that for both the embodiments of FIGS. 3 and 4, an optional feature is that the disease detection module 42 generates a 3D spatial map 74, 75 of the disease region within the 3D field of the image frame corresponding to the disease feature for at least one input 3D image frame. This may be a map of the segmented region 75 in the case of the embodiment of FIG. 4, or a saliency map 74 in the case of the embodiment of FIG. 3.

[0060] The output of the disease detection module 42 is supplied to the planar mapping module 44.

[0061] The planar mapping module 44 performs an optimization step of calculating an optimal set of one or more 2D slices through the 3D field of the image frame in order to image one or more disease regions associated with the detected one or more disease features. In other words, an optimal acquisition protocol is determined that includes a plurality of 2D slices and, optionally, beamforming parameters and frame rates that are most suitable for verifying any potential findings detected by the disease detection module.

[0062] In this example, the planar mapping module 44 is adapted to receive, as input, a 3D map of the detected disease region within the 3D field and is adapted to determine a set of one or more 2D slices through the 3D field that intersects the disease region, depending on the map.

[0063] When the saliency map 74 (FIG. 3) is generated, the saliency map can be used as a 3D spatial map of the disease region. For example, one approach is to calculate the slices that best cover all regions with a high saliency response. In other words, the planar mapping module 44 is adapted to determine a set of one or more 2D slices based on the plane of maximum saliency through the saliency map. The plane of maximum saliency means, for example, the plane that plots the path of the plane of maximum saliency, i.e., the plane that contains / covers the maximum aggregated saliency.

[0064] Optionally, the determined plane can be determined to satisfy one or more additional constraints. Referring to the one or more additional constraints, these can depend in part on the particular disease characteristics to which the disease region corresponds. The one or more additional constraints can comprise requirements for minimizing the number of 2D slices while still intersecting all of the regions. The one or more additional constraints can comprise one or more tolerances for the 2D slice surface orientation with respect to one or more directions.

[0065] Exemplary implementations in some examples use weighted saliency responses and can determine the closest standard, user-defined, or automatically defined anatomical plane to derive the optimal slices. The automatically defined anatomical plane can be derived, as described above, by fitting the plane through maximum saliency (e.g., maximum intensity projection aligned along a particular anatomical landmark).

[0066] When the disease detection module 42 generates a 3D spatial map of the segmented region 75 (Figure 4), a 2D plane can be derived using the 3D spatial map of the segmented suspected region. For example, one approach is for the plane mapping module to perform a spatial fitting of the plane to one or more disease regions within the map, determine a set of one or more 2D slices that intersect all of the regions, and optionally satisfy one or more additional constraints (see the discussion above).

[0067] When a 3D map of the disease area is not generated (i.e., different from the embodiments of FIGS. 3 and 4), another approach for determining a plane may be to utilize guidelines or protocols derived from a database or a prediction model, i.e., to select the anatomical plane that is typically most suitable for detecting a disease that may be associated with the detected disease feature. For example, the plane mapping module 44 may be adapted to use a look-up table to select a predetermined plane based on one or more detected disease features.

[0068] In both the method of FIG. 3 and the method of FIG. 4, after determining the 2D slices, the method further includes generating control instructions 18 for output to the ultrasound acquisition system to effect automatic acquisition of the determined set of one or more 2D slices, for example using B-mode images. These control instructions may be generated by the plane mapping module 44 or by another module or routine of one or more processors of the processing device 32 implementing the method. The ultrasound acquisition system receives the instructions and acquires the 2D slices in accordance with the instructions (82).

[0069] The input to the ultrasound acquisition system is a set of control instructions. At the time of receiving the control instructions, the ultrasound acquisition system may still be used by an operator to acquire image data. Optionally, the acquisition of new 2D slices (e.g., B-mode) may be performed in the background, and optionally, a prompt presented on the user interface display may be used to notify the user that additional acquisition is in progress and thus the probe needs to be held in a predetermined position. In some examples, the control instructions may be adapted to cause the ultrasound acquisition system to interleave the acquisition of a set of one or more 2D slices with any other acquisition sequence that the ultrasound acquisition system is currently executing.

[0070] In some examples, after determining a set of 2D slices, the method further includes controlling a user interface to generate a user-recognizable prompt that requests approval to obtain the set of 2D slices, and the generation of the control instructions is only executed in response to receiving a user input indicating approval from the user interface. This can be generated on the user interface of the ultrasound acquisition system in some examples. For example, the recording of additional slices can be triggered by a user interface event and an optional preview of the proposed acquisition protocol or pictogram. This is, for example, a dialog box asking the operator for permission to acquire slices, and an obligatory user input in the form of pressing an OK button, a foot pedal, a button, etc., which cancels the acquisition of additional slices.

[0071] Regarding the generation of control instructions leading to the acquisition of a specifically derived set of 2D planes, a dedicated module can be included in the processing device for the purpose of generating such control instructions. The control instructions can indicate the acquisition parameters to be used by the ultrasound acquisition system to acquire one or more 2D slices. For example, these can include beamforming parameters. They can also include other parameters that specify, for example, one or more of scan line density, transmit frequency, and receive filtering to optimize the image quality of the desired scan region.

[0072] Of course, the ultrasound imaging system 52 can typically include a local beamform controller operable to control beamform parameters to acquire one or more selected planes, typically defined with respect to the coordinate system of the field of view of the imaging probe, for example. Thus, in some examples, the control instructions can simply provide an indication of one or more planes within the imaging field of view to be acquired, and the necessary acquisition parameters including the beamforming parameters for acquiring these planes are determined locally by the ultrasound acquisition system 52.

[0073] Accordingly, the ultrasonic acquisition system receives an instruction and acquires a 2D slice 82 according to the instruction.

[0074] Preferably, the control instruction is adapted to control the acquisition of 2D slices having a higher spatial resolution than the input 3D ultrasonic image data. To achieve a minimum temporal resolution, the 3D image typically has its spatial resolution limited for each 3D frame. Thus, by acquiring 2D images of each suspicious disease region, diagnostic analysis can be performed using higher spatial resolution images.

[0075] Optionally, an additional step can be implemented to check or verify (84) the acquired slice against the planned optimal slice determined by the planar mapping module 44 and whose index is output from the planar mapping module.

[0076] This can utilize a verification module that performs a comparison between a given newly acquired 2D image and a 2D slice extracted from 3D imaging data corresponding to the determined 2D plane that is intended to be acquired. Thus, it can be determined whether the acquired image actually matches the intended plane to be imaged. For example, the patient may move between determining the plane and capturing the 2D image, which can lead to the acquired image frame not matching the intended slice through the anatomical structure.

[0077] For example, the comparison can be made by calculating the similarity between each slice from a 3D volume (having, for example, a coarser resolution) and the acquired 2D image (having, for example, a finer resolution). The same features should appear in both images, for example, the same anatomical landmarks should be present at common positions and orientations. In other words, in some examples, once the 2D slices to be acquired are determined, corresponding 2D slices through the 3D data set can be extracted (using, for example, a multi-planar format), and then these can be compared with the newly acquired 2D image to check the degree of match between the two.

[0078] Additionally or alternatively, the verification may utilize, for example, a plane prediction module that is applied to the acquired 2D images generated by an ultrasound acquisition system and, as output, each acquired 2D image is adapted to generate a prediction of the corresponding anatomical plane. This may be, for example, a plane regression module having the task of estimating a given acquired 2D image, the plane of which represents the image through the imaged anatomical structure. The results of the plane prediction module can then be compared with the planned or intended 2D slices, and thus it can be determined whether the acquired images actually match the intended planes to be imaged.

[0079] For example, the plane prediction module can be implemented by training a regression CNN using pairs of 2D images and their respective known plane parameters. The inventors have found that such predictors can be further stabilized when, in training, the CNN is trained on the detected anatomical contours instead of using the entire 2D image. This effectively creates a handmade feature bottleneck. An exemplary algorithm following this approach (in slightly different regions of skeletal x-rays) is published as Kronke et al., "CNN-based pose estimation for evaluating the quality of ankle x-ray images" (SPIE Medical Imaging 2022).

[0080] In a further step, the method includes controlling the user interface to generate a visual output 86 that preferably represents a set of one or more 2D slices that have been obtained. These can be visualized, for example, immediately or in response to a request by the user.

[0081] In some embodiments, optionally, in addition to determining one or more slices that help support or confirm a suspicious disease finding, one or more slices that may contradict or negate the suspicion can be obtained. This can improve the confidence in the diagnosis that has an unbiased view of the suspicious finding.

[0082] In some embodiments, an additional step can be performed where the acquired 2D image slices are supplied as further input to the disease detection module 42. The result can help confirm or strengthen previous disease detection findings. For example, if the result of a further application of the disease detection module is a negative finding (no disease features are detected), the acquired image slices can be discarded. This reduces false detections.

[0083] In some embodiments, the one or more planes obtained for one or more of the possible disease features can include a standardized plane that has been pre - determined to represent an optimal or optimized anatomical view for analyzing a given disease feature. A look - up table can be used to look up the standard plane for a given feature. One way to generate the standard plane can be to perform a training phase where a pre - trained machine learning module receives, as input, a plurality of saliency - map - derived planes (from different examinations and patients) and delivers, as output, a rule - based plane mapping by, for example, selecting three plane - defining landmarks. This embodiment provides an opportunity to improve the general plane portfolio. For a particular disease, for example, a new "standard" plane can be defined.

[0084] One particularly advantageous application of an embodiment of the present invention is in the field of cardiac imaging for ultrasonic examination of the human heart. However, this application is not at all limited to this field. The present invention is generally applicable to any imaging modality. It can find particularly advantageous applications for non-ionizing imaging modalities. It can find particularly advantageous applications for imaging modalities that enable the acquisition of images with different levels of quality / resolution using certain geometric settings, such as optical coherence tomography (OCT). Furthermore, the body part of interest can be any part of the human body and is not limited to the heart. The present invention can be applied, for example, as non-limiting examples, to fetal scans, pathology, or veterinary applications.

[0085] As described above, the present invention can be implemented in software form. Thus, another aspect of the present invention is a computer program product comprising computer program code or other code means configured to cause a processor (e.g., operably coupled to an ultrasonic acquisition system) to execute a method according to an example or embodiment of the present invention described in this document or according to any claim of this patent application when executed on the processor.

[0086] The above-described embodiments of the present invention use a processing device. The processing device generally may comprise a single processor or a plurality of processors. It may be arranged within a single housing device, structure, or unit, or may be distributed among a plurality of different devices, structures, or units. Thus, a reference to a processing device adapted or configured to perform a particular step or task may correspond to that step or task being performed by any one or more of a plurality of processing components, alone or in combination. Those skilled in the art will understand how such distributed processing devices can be implemented. The processing device may include a communication module or input / output section for receiving data and outputting the data to further components.

[0087] One or more processors of a processing device can be implemented in a number of ways using software and / or hardware to perform the various functions required. A processor typically uses one or more microprocessors that can be programmed using software (e.g., microcode) to perform the required functions. A processor can be implemented as a combination of dedicated hardware for performing some functions and one or more programmed microprocessors and associated circuitry for performing other functions.

[0088] Examples of circuits that can be used in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field programmable gate arrays (FPGAs).

[0089] In various implementations, a processor can be associated with one or more storage media such as volatile and non-volatile computer memories such as RAM, PROM, EPROM, and EEPROM. The storage media can be encoded with one or more programs that perform the required functions when executed on one or more processors and / or controllers. Various storage media can be fixed within the processor or controller or can be portable such that one or more programs stored thereon can be loaded into the processor.

[0090] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality.

[0091] A single processor or other unit can fulfill the functions of several items recited in the claims.

[0092] The mere fact that certain means are recited in mutually different dependent claims does not indicate that a combination of these means cannot be used advantageously.

[0093] The computer program may be stored / distributed on a suitable medium such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms such as the Internet or other wired or wireless electrical communication systems.

[0094] It should be noted that when the term "adapted to" is used in the claims or the description, the term "adapted to" is intended to be equivalent to the term "configured to".

[0095] Any reference signs in the claims shall not be construed as limiting the scope.

Claims

1. A computer implementation method, The steps include receiving a 3D ultrasound image dataset having at least one 3D image frame spanning a 3D field as input from an ultrasound acquisition system, The steps include processing an input 3D image frame and applying a disease detection module adapted to detect one or more suspicious disease features from a predefined set of possible disease features, The steps include applying a planar mapping module to each 3D image frame, which is adapted to determine one or more sets of 2D slices through the 3D field of the image frame, based on the output from the disease detection module, in order to image one or more disease regions associated with the one or more detected disease features, The steps include generating control commands to output to the ultrasound acquisition system in order to bring about the automatic acquisition of a new set of 2D image frames corresponding to the set of one or more 2D slices, and A method having

2. The method according to claim 1, wherein the input 3D ultrasound image dataset has a stream of 3D image frames, and the steps of the method are performed in real time with the reception of each 3D image frame.

3. The method according to claim 1, wherein the plane mapping module is adapted to select a predetermined plane based on the one or more detected disease features using a lookup table.

4. The disease detection module is adapted to generate a 3D spatial map of the disease region within the 3D field of the image frame corresponding to the disease feature for at least one 3D image frame. The method according to claim 1, wherein the planar mapping module is adapted to receive, as input, a 3D map of a detected disease area in a 3D field, and, depending on the map, to determine a set of one or more 2D slices through the 3D field intersecting the disease area.

5. The method according to claim 4, wherein the planar mapping module is adapted to perform spatial fitting of a plane to the one or more disease regions and to determine one or more sets of 2D slices that intersect all of the regions and optionally satisfy one or more further constraints.

6. The method according to claim 4, wherein, for at least one subset of a predefined set of disease features, the detection of disease features by the disease detection module includes the step of segmenting and classifying one or more spatial regions as suspicious regions, and the 3D spatial map output by the disease detection module has a map of the segmented suspicious regions.

7. For at least one subset of the predefined set of disease features, the detection of each disease feature by the disease detection module comprises the steps of: calculating a 3D sampling map across a 3D field in relation to the disease feature; and deriving individual classifications of 3D images related to the feature based on the sampling map, wherein the sampling map is used as a 3D spatial map of the disease region. The planar mapping module is adapted to determine the set of one or more 2D slices based on the fitting plane of maximum splendor that passes through the splendor map. The method according to claim 4.

8. The method according to claim 1, wherein the control command is adapted to cause the ultrasonic acquisition system to interleave the acquisition of the new set of 2D image frames with any other acquisition sequence currently being performed by the ultrasonic acquisition system.

9. The method according to claim 1, wherein the disease detection module has at least one trained machine learning algorithm, preferably a convolutional neural network (CNN).

10. The method according to claim 1, wherein the control command is adapted to control the acquisition of a new 2D image frame having a higher spatial resolution than the input 3D ultrasound image data.

11. The method according to claim 1, further comprising the steps of receiving the acquired set of new 2D image frames and controlling a user interface to generate a visual output representing them.

12. The method according to claim 1, further comprising the step of controlling a user interface to obtain the new set of 2D image frames by generating a user-recognizable prompt requesting approval after determining the set of 2D slices, wherein the generation of the control command is performed only in response to the receipt from the user interface of user input indicating approval.

13. A computer program product having coding means configured to cause a processor operably coupled with an ultrasonic acquisition system to perform the method according to any one of claims 1 to 12 when executed on the processor.

14. Apparatus, Input / Output section, One or more processors, The input / output unit includes the step of receiving a 3D ultrasound image dataset, which comprises at least one 3D image frame spanning a 3D field, as input from the ultrasound acquisition system. The steps include processing an input 3D image frame and applying a disease detection module adapted to detect one or more suspicious disease features from a predefined set of possible disease features, The steps include applying a planar mapping module adapted to determine one or more sets of 2D slices through the 3D field of each image, based on the output from the disease detection module, in order to image one or more disease regions associated with the one or more disease features detected, The steps include generating control commands to be output to the ultrasound acquisition system via the input / output unit, in order to bring about the automatic acquisition of a new set of 2D image frames corresponding to the set of one or more 2D slices, and A processor and configured to perform a method having A processing apparatus having

15. An ultrasonic acquisition system, The processing apparatus according to claim 14, which is operably coupled to the ultrasonic acquisition system, A system equipped with these features.

Images