Method and system for determining supplemental ultrasound views

Abstract

The present invention provides an ultrasound imaging method for determining a supplementary view of interest based on abnormal features identified in a region of interest of an ultrasound image. The method includes: obtaining an ultrasound image of a region of interest of an object; and identifying abnormal features within the region. The identified abnormal features can then be used to determine one or more available supplementary ultrasound images of interest to the object. The one or more available supplementary ultrasound images can then be displayed to a user, and the supplementary ultrasound view to be viewed can then be selected by the user from the displayed available supplementary ultrasound images.

Description

Technical Field

[0001] The present invention relates to the field of ultrasound, and in particular to the field of ultrasound imaging. Background Art

[0002] Ultrasound imaging is used in a variety of medical imaging applications. When performing ultrasound imaging, it is possible that abnormalities can be detected, which may indicate an underlying medical condition.

[0003] A common application of ultrasound imaging is a fetal screening exam, in which the health of the fetus is assessed during a given stage in pregnancy. During a screening exam, the clinician performs measurements of various specific anatomical structures and compares the values ​​to a common measurement table. When the measurement results are not within the normal range, additional examinations and additional measurements are ordered to confirm that the abnormality is a cause for concern. The additional examinations and measurements often vary depending on the suspected pathology.

[0004] Manually selecting appropriate additional views to observe the anatomical structures and manually performing measurements can be long and difficult, especially if several anatomical regions need to be examined.

[0005] Therefore, there is a need to provide a method for obtaining additional ultrasound views of an imaging region in a more efficient manner without requiring a large amount of additional hardware.

[0006] EP 2918233 discloses an ultrasound diagnostic apparatus including: a display unit displaying a first screen including at least one mark; a user interface unit for receiving a selection of a predetermined mark; and a display unit displaying the first screen including a second ultrasound image having a view of an object changed according to the predetermined mark. Summary of the invention

[0007] The invention is defined by the claims.

[0008] According to an example of an aspect of the present invention, there is provided a computer-implemented ultrasound imaging method, the method comprising:

[0009] obtaining an ultrasound image containing a region of interest of the object;

[0010] analyzing one or more features of the ultrasound image;

[0011] identifying abnormal features of the region of interest based on the analysis of the one or more features;

[0012] automatically determining one or more available supplemental ultrasound images of interest for the subject based on the identified abnormal features; and

[0013] The one or more available supplemental ultrasound images are displayed to a user.

[0014] The method provides for automatic acquisition of ultrasound images associated with abnormal features identified in a region of interest, which are complementary to the originally acquired ultrasound images.

[0015] For example, an ultrasound image may contain abnormal features that require additional investigation. In order to perform the investigation, multiple supplementary views may be required to accurately identify pathology associated with the abnormal features. The supplementary views may be automatically determined without requiring additional input by the user.

[0016] The selection of the desired supplemental image can be performed by the user from a list of one or more available supplemental ultrasound images automatically selected based on the abnormality characteristics, or can be automatically performed by finding a supplemental image among images previously acquired during the examination. The user can then select and view an image among the selected views.

[0017] In an arrangement, the ultrasound image comprises a 3D ultrasound image, and wherein the determination of the one or more available supplementary ultrasound images comprises obtaining one or more supplementary 2D views from the 3D ultrasound image.

[0018] In this way, the supplementary ultrasound image may be obtained directly from the initially acquired 3D ultrasound image.

[0019] In an embodiment, the identifying of abnormal features of the region of interest comprises performing organ detection.

[0020] By detecting organs within the ultrasound image, it is possible to identify abnormal features within a specific target region. For example, the fetal head is an important ethmoid region. By performing skull detection as organ detection, it is possible to automatically locate the fetal head within the ultrasound image.

[0021] In an embodiment, the region of interest comprises a portion of a fetus.

[0022] In this way, the process of fetal screening is simplified and accelerated.

[0023] In an arrangement, the identifying of the abnormal features of the region of interest includes performing skeletal and mid-sagittal brain plane detection on the fetus.

[0024] The midsagittal brain plane (also referred to as the median plane) bisects the fetus to provide a profile cross-sectional image. This view can then be used to identify any relevant abnormal features of the fetus, and in particular specific abnormal features within the head of the fetus.

[0025] In another arrangement, the mid-sagittal brain plane detection includes:

[0026] Perform multiresolution analysis; and

[0027] Perform Hough detection.

[0028] In an embodiment, the identifying of the abnormal features of the region of interest comprises:

[0029] extracting a contour of the fetus; and

[0030] Based on the contour of the fetus, obtaining:

[0031] naso-chin angle; or

[0032] Lower facial angle.

[0033] In this way, the facial angles of the fetus can be used to determine abnormal facial formation if the angles are not within a typical range.

[0034] In an embodiment, the determination of one or more available supplemental ultrasound images comprises automatically generating a list of possible supplemental images based on the identified abnormality features.

[0035] In this manner, all possible supplemental views relevant to the investigation of a given anomaly feature may be considered, thereby providing multiple alternative options when a given view is obstructed or difficult to obtain.

[0036] In further embodiments, said determining of said one or more available supplemental ultrasound images further comprises:

[0037] comparing the list of possible supplemental images to previously acquired ultrasound images of the subject; and

[0038] A previously acquired ultrasound image is selected as an available supplementary ultrasound image if the previously acquired ultrasound image matches a possible supplementary view.

[0039] In this way, previously acquired ultrasound images of the subject may be used to provide the required supplemental views.

[0040] In a further arrangement, the obtaining of the one or more available supplemental ultrasound images from the 3D ultrasound image comprises:

[0041] Extract anatomical reference structures;

[0042] A set of supplementary 2D views is obtained by extracting one or more 2D planes of interest from the 3D ultrasound image based on the extracted anatomical reference structure.

[0043] By extracting anatomical reference structures as reference points, it is possible to navigate in the 3D image and extract relevant 2D views of interest from the 3D image.

[0044] For example, the cervical region or neck of the fetus can be used to capture a 2D view of the lower head, such as the jaw and mouth, which may be of interest for screening of some pathologies.

[0045] In yet another arrangement, said obtaining of said one or more available supplemental ultrasound images from said 3D ultrasound image comprises:

[0046] extracting structural features from the anatomical reference structure;

[0047] determining whether the anatomical reference structure can be tracked in the 3D ultrasound image based on the structural feature;

[0048] In case the anatomical reference structure can be tracked in the 3D ultrasound image:

[0049] A 2D plane of interest is extracted from the 3D ultrasound image by using the anatomical reference structure as an anatomical landmark.

[0050] In this way, specific structural features can be used to capture desired 2D planes of interest from 3D images.In particular, structural features related to specific pathologies can be exploited when said pathology is being investigated.

[0051] In an embodiment, the anatomical reference structure comprises one or more of:

[0052] organs; and

[0053] Bone structure.

[0054] In this way, it is possible to include views containing information related to a wide variety of different pathologies.

[0055] In embodiments, the organ comprises one or more of: heart; kidney; bladder; stomach; and the skeletal structure comprises one or more of: skull; spine; thorax; femur; and humerus.

[0056] According to an example of an aspect of the invention, there is provided a computer program comprising computer program code means adapted to implement the method described above when the computer program is run on a computer.

[0057] According to an example of an aspect of the present invention, there is provided an ultrasound imaging system, the system comprising:

[0058] an ultrasound probe adapted to obtain an ultrasound image of a region of interest containing an object; and a processor adapted to:

[0059] analyzing one or more features of the ultrasound image;

[0060] identifying abnormal features of the region of interest based on the analysis of the one or more features;

[0061] automatically determining one or more available supplemental ultrasound images based on the identified abnormal features; and

[0062] The one or more available supplemental ultrasound images are displayed to a user.

[0063] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Examples of the present invention will now be described in detail with reference to the accompanying drawings, in which:

[0065] Figure 1 An ultrasonic diagnostic imaging system is shown explaining general operation;

[0066] Figure 2 The method of the present invention is shown;

[0067] Figure 3 Another method of the present invention is shown;

[0068] Figure 4 A schematic midsagittal plane view of the fetal head is shown;

[0069] Figure 5 showing example images of 2D view planes orthogonal to fetal spine extraction; and

[0070] Figure 6 A fetal image slice is shown with respect to the fetal spine. DETAILED DESCRIPTION

[0071] The present invention will be described with reference to the accompanying drawings.

[0072] It should be understood that the detailed description and specific examples, although indicating exemplary embodiments of the device, system and method, are intended only for illustrative purposes and are not intended to limit the scope of the invention. These and other features, aspects and advantages of the device, system and method of the present invention will become better understood based on the following description, the appended claims and the accompanying drawings. It should be understood that the drawings are merely schematic and are not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to indicate the same or similar parts.

[0073] The present invention provides an ultrasound imaging method for determining a supplementary view of interest based on abnormal features identified in a region of interest of an ultrasound image. The method includes: obtaining an ultrasound image of a region of interest of an object; and identifying abnormal features within the region. The identified abnormal features can then be used to determine one or more available supplementary ultrasound images of interest for the object. One or more available supplementary ultrasound images can then be displayed to a user, and the supplementary ultrasound view to be viewed can then be selected by the user from the displayed available supplementary ultrasound images.

[0074] First, refer to Figure 1 The general operation of an exemplary ultrasound system is described herein with emphasis placed on the image processing functionality of the system as the present invention relates to the processing of ultrasound images acquired by the system.

[0075] The system includes an array transducer probe 4 having a transducer array 6 for transmitting ultrasound and receiving echo information. The transducer array 6 may include CMUT transducers; piezoelectric transducers formed of materials such as PZT or PVDF; or any other suitable transducer technology. In this example, the transducer array 6 is a two-dimensional array of transducers 8 capable of scanning a 2D plane or a three-dimensional volume of a region of interest. In another example, the transducer array may be a 1D array.

[0076] The transducer array 6 is coupled to a microbeamformer 12 which controls the reception of signals by the transducer elements. The microbeamformer is capable of at least partially beamforming signals received by sub-arrays of transducers (often referred to as "groups" or "tiles"), as described in U.S. Pat. Nos. 5,997,479 (Savord et al.), 6,013,032 (Savord), and 6,623,432 (Powers et al.).

[0077] It should be noted that the microbeamformer is entirely optional. In addition, the system includes a transmit / receive (T / R) switch 16 to which the microbeamformer 12 may be coupled and which switches the array between transmit and receive modes and protects the main beamformer 20 from high energy transmit signals when the microbeamformer is not used and the transducer array is operated directly by the main system beamformer. Transmission of ultrasound beams from the transducer array 6 is directed by a transducer controller 18 coupled to the microbeamformer through the T / R switch 16 and a main transmit beamformer (not shown), which may receive input from a user operating a user interface or control panel 38. The controller 18 may include transmit circuitry arranged to drive the transducer elements of the array 6 (directly or via the microbeamformer) during transmit mode.

[0078] In a typical line-by-line imaging sequence, the beamforming system within the probe may operate as follows. During transmit, the beamformer (which may be a microbeamformer or a main system beamformer, depending on the implementation) activates the transducer array or a subaperture of the transducer array. A subaperture may be a one-dimensional line of transducers or a two-dimensional patch of transducers within a larger array. In transmit mode, the focusing and steering of the ultrasound beam generated by the array or a subaperture of the array is controlled as described below.

[0079] After receiving the backscattered echo signal from the object, the received signal undergoes receive beamforming (described below) to align the received signal and, in the case of using sub-apertures, the sub-aperture is then shifted, for example, by one transducer element. The shifted sub-aperture is then activated, and the process is repeated until all transducer elements of the transducer array have been activated.

[0080] For each line (or sub-aperture), the total received signal for the associated line used to form the final ultrasound image will be the sum of the voltage signals measured by the transducer elements of a given sub-aperture during the receive period. After the beamforming process below, the resulting line signal is typically referred to as radio frequency (RF) data. Each line signal (RF data set) generated by the respective sub-aperture then undergoes additional processing to generate the line of the final ultrasound image. The variation in the amplitude of the line signal over time will contribute to the variation in the brightness of the ultrasound image with depth, where high amplitude peaks will correspond to bright pixels (or sets of pixels) in the final image. Peaks that appear near the beginning of the line signal will represent echoes from shallow structures, while peaks that gradually appear later in the line signal will represent echoes from structures at increasing depths within the object.

[0081] One of the functions controlled by the transducer controller 18 is the direction of beam steering and focusing. The beam can be steered straight ahead (orthogonal to) the transducer array, or at different angles for a wider field of view. The steering and focusing of the transmit beam can be controlled based on the transducer element actuation time.

[0082] In general ultrasound data acquisition two methods can be distinguished: plane wave imaging and "beam steering" imaging. The two methods are distinguished by the presence of beam forming in the transmit mode ("beam steering" imaging) and / or in the receive mode (plane wave imaging and "beam steering" imaging).

[0083] Looking first at the focusing function, by activating all transducer elements simultaneously, the transducer array generates a plane wave that diverges as it travels through the object. In this case, the beam of ultrasound waves remains unfocused. By introducing a position-dependent time delay to the activation of the transducers, the wavefront of the beam can be focused at a desired point, which is called the focal zone. The focal zone is defined as the point where the lateral beamwidth is less than half the transmit beamwidth. In this way, the lateral resolution of the final ultrasound image is improved.

[0084] For example, if the time delay causes the transducer elements to be activated in series starting with the outermost elements and ending at the center element(s) of the transducer array, a focal zone will be formed at a given distance from the probe, coinciding with the center element(s). The distance of the focal zone from the probe will vary depending on the time delay between each subsequent round of transducer element activation. After the beam passes through the focal zone, it will begin to diverge, forming a far-field imaging region. It should be noted that for a focal zone positioned close to the transducer array, the ultrasound beam will diverge rapidly in the far field, resulting in beam width artifacts in the final image. Typically, the near field located between the transducer array and the focal zone shows little detail due to the large overlap in the ultrasound beam. Therefore, changing the position of the focal zone can result in significant changes in the quality of the final image.

[0085] It should be noted that in transmit mode, unless the ultrasound image is divided into multiple focal zones (each of which may have a different transmit focus), only one focus may be defined.

[0086] Additionally, after receiving the echo signal from within the object, the inverse of the above process can be performed to perform receive focusing. In other words, the incoming signal can be received by the transducer elements and undergo an electronic time delay before being passed into the system for signal processing. The simplest example of this is called delay-and-sum beamforming. The receive focus of the transducer array can be adjusted dynamically based on time.

[0087] Turning now to the function of beam steering, by properly applying time delays to the transducer elements, it is possible to impart a desired angle on the ultrasound beam as it leaves the transducer array. For example, by activating the transducers on a first side of the transducer array in a sequence ending on opposite sides of the array, followed by the remaining transducers, the wavefront of the beam will be angled towards the second side. The magnitude of the steering angle relative to the normal to the transducer array depends on the magnitude of the time delay between subsequent transducer element activations.

[0088] Furthermore, the steered beam can be focused, wherein the total time delay applied to each transducer element is the sum of both the focusing and steering time delays. In this case, the transducer array is called a phased array.

[0089] In the case of CMUT transducers that require a DC bias voltage for activation thereof, the transducer controller 18 may be coupled to control the transducer array's DC bias control 45. The DC bias control 45 sets the DC bias voltage(s) applied to the CMUT transducer elements.

[0090] For each transducer element of the transducer array, an analog ultrasound signal, generally referred to as channel data, enters the system through a receive channel. In the receive channel, a partial beamformed signal is generated by the microbeamformer 12 from the channel data and then passed to the main receive beamformer 20, where the partial beamformed signals from the individual tiles of the transducers are combined into a fully beamformed signal, referred to as radio frequency (RF) data. The beamforming performed at each stage may be performed as described above, or may include additional functionality. For example, the main beamformer 20 may have 128 channels, each of which receives a partial beamformed signal from a tile of tens or hundreds of transducer elements. In this way, the signals received by thousands of transducers of the transducer array may effectively contribute to a single beamformed signal.

[0091] The beamformed receive signals are coupled to a signal processor 22. The signal processor 22 can process the received echo signals in various ways, such as bandpass filtering; decimation; I and Q component separation; and harmonic signal separation, which is used to separate linear signals from nonlinear signals, thereby enabling the identification of nonlinear (higher harmonics of the fundamental frequency) echo signals returned from tissue and microbubbles. The signal processor 22 can also perform additional signal enhancement, such as speckle reduction, signal synthesis, and noise removal. The bandpass filter in the signal processor can be a tracking filter, the passband of which slides from a higher frequency band to a lower frequency band when the echo signals are received from increasing depths, thereby rejecting noise at higher frequencies from greater depths, which generally lack anatomical information.

[0092] The beamformers for transmission and for reception are implemented in different hardware and may have different functions. Of course, the receiver beamformer is designed to take into account the characteristics of the transmit beamformer. Figure 1 Only the receiver beamformers 12, 20 are shown in FIG. In the overall system there will also be a transmit chain with a transmit microbeamformer and a main transmit beamformer.

[0093] The function of the microbeamformer 12 is to provide initial combining of the signals in order to reduce the number of analog signal paths. This is typically performed in the analog domain.

[0094] Final beamforming is done in the main beamformer 20 and is typically done after digitization.

[0095] The transmit and receive channels use the same transducer array 6 with fixed frequency bands. However, the bandwidth occupied by the transmit pulses can vary depending on the transmit beamforming used. The receive channel can capture the entire transducer bandwidth (which is the classical approach) or by using bandpass processing, which can only extract the bandwidth containing the desired information (e.g. harmonics of the main harmonic).

[0096] The RF signal can then be coupled to a B-mode (i.e., brightness mode or 2D imaging mode) processor 26 and a Doppler processor 28. The B-mode processor 26 performs amplitude detection on the received ultrasound signal to image structures in the body, such as organ tissue and blood vessels. In the case of line-by-line imaging, each line (beam) is represented by an associated RF signal, whose amplitude is used to generate a brightness value to be assigned to a pixel in the B-mode image. The exact position of the pixel within the image is determined by the position of the associated amplitude measurement along the RF signal and the number of lines (beams) of the RF signal. B-mode images of such structures can be formed in harmonic or fundamental image modes or a combination of both, as described in U.S. Pat. No. 6,283,919 (Roundhill et al.) and U.S. Pat. No. 6,458,083 (Jago et al.). The Doppler processor 28 processes the temporally different signals generated by tissue movement and blood flow to detect moving matter, such as the flow of blood cells in the image field. The Doppler processor 28 typically includes a wall filter having parameters set to pass or reject echoes returned from selected types of materials within the body.

[0097] The structure and motion signals generated by the B-mode and Doppler processors are coupled to the scan converter 32 and the multi-plane reformatter 44. The scan converter 32 arranges the echo signals in a spatial relationship, and the echo signals are received in a desired image format according to the spatial relationship. In other words, the scan converter is used to convert the RF data from a cylindrical coordinate system to a Cartesian coordinate system suitable for displaying an ultrasound image on the image display 40. In the case of B-mode imaging, the brightness of a pixel at a given coordinate is proportional to the amplitude of the RF signal received from the position. For example, the scan converter can arrange the echo signals into a two-dimensional (2D) fan format or a pyramidal three-dimensional (3D) image. The scan converter can superimpose a color corresponding to the motion at a point in the image field to the B-mode structural image, where the Doppler estimated velocity produces a given color. The combined B-mode structural image and color Doppler image depict the motion of tissue and blood flow within the structural image field. As described in U.S. Pat. No. 6,443,896 (Detmer), the multi-plane reformatter converts the echoes received from points in a common plane in a volumetric region of the body into an ultrasound image of the plane. The volume renderer 42 converts the echo signals of the 3D data set into a projected 3D image as viewed from a given reference point, as described in US Pat. No. 6,530,885 (Entrekin et al.).

[0098] The 2D or 3D images are coupled from the scan converter 32, the multiplanar reformatter 44, and the volume renderer 42 to the image processor 30 for further enhancement, buffering, and temporary storage for display on the image display 40. The imaging processor may be adapted to remove certain imaging artifacts from the final ultrasound image, such as: acoustic shadowing, for example, caused by strong attenuators or refraction; post-enhancement, for example, caused by weak attenuators; reverberation artifacts, for example, where highly reflective tissue interfaces are located in close proximity; etc. In addition, the image processor may be adapted to process certain speckle reduction functions in order to improve the contrast of the final ultrasound image.

[0099] In addition to being used for imaging, the blood flow values ​​produced by the Doppler processor 28 and the tissue structure information produced by the B-mode processor 26 are also coupled to a quantification processor 34. The quantification processor produces measurements of various flow conditions, such as volumetric rate of blood flow in addition to structural measurements such as organ size and fetal age. The quantification processor can receive input from a user control panel 38, such as the point in the anatomical structure of the image at which the measurement is to be made.

[0100] The output data from the quantization processor is coupled to a graphics processor 36 for reproducing the measurement graphs and values ​​on a display 40 together with the image, and for audio output from the display device 40. The graphics processor 36 may also generate graphic overlays for display with the ultrasound image. These graphic overlays may contain standard identification information, such as the patient's name, the date and time of the image, imaging parameters, etc. For these purposes, the graphics processor receives input from a user interface 38, such as the patient's name. The user interface is also coupled to the transmit controller 18 to control the generation of ultrasound signals from the transducer array 6, and therefore the images produced by the transducer array and the ultrasound system. The transmit control function of the controller 18 is only one of the functions performed. The controller 18 also takes into account the operating mode (given by the user) and the corresponding desired transmitter configuration and bandpass configuration in the receiver analog-to-digital converter. The controller 18 may be a state machine with fixed states.

[0101] The user interface is also coupled to a multi-planar reformatter 44 for selecting and controlling the planes of a plurality of multi-planar reformatted (MPR) images, which may be used to perform quantitative measurements in the image fields of the MPR images.

[0102] Figure 2 A method 100 of the present invention is shown.

[0103] The method begins in step 110 by obtaining an ultrasound image containing a region of interest of a subject.

[0104] The subject may be, for example, a patient undergoing ultrasound screening, in which case the region of interest may be the abdominal region. Furthermore, the region of interest may include at least part of a fetus, in which case the ultrasound imaging method may be used for fetal screening.

[0105] Many fetal pathologies are associated with various identifiable features or symptoms. For example, symptoms of trisomy 13 (occurring in one in 6500 fetuses) can include: craniofacial defects; brain malformations; urogenital tract abnormalities; and less frequently, heart and finger defects as described in "Ultrasound features in trisomy 13 (Patau syndrome) and trisomy 18 (Edwards syndrome) in a consecutive series of 47 cases." (Kroes, I., Janssens, S. and Defoort, P. Facts Views Vis Obgyn. 2014; 6(4): 245–249).

[0106] In step 120, abnormal features of the region of interest are identified.

[0107] Abnormal features of the region of interest can be automatically identified using appropriate automated image analysis techniques, such as feature extraction. For example, automated feature detection can be used for detection of conditions such as ventriculomegaly in the fetal brain.

[0108] Alternatively, the abnormal feature can be manually identified by the user from the acquired ultrasound image. In this case, the user can indicate the identified abnormality to the ultrasound system by selecting the abnormal feature from a list of known abnormal features.

[0109] By way of example, the identified feature may be micrognathia (a craniofacial abnormality corresponding to a small jaw size relative to the head). According to Prenatal Diagnosis of Orofacial Malformations. Tonni G., Sepulveda, W. and Wong, AE Springer, 2017: "Micrognathia […] is rarely isolated and is often associated with other anomalies, such hemifacial macrosomia, Pierre Robin sequence, and Treacher Collins syndrome […]. Chromosomal abnormalities have been reported in 66% of fetuses with micrognathia, commonly in triploidy, trisomy 13, and trisomy 18."

[0110] Abnormal features can be detected either because measurements obtained from the ultrasound image (either automatically or manually by the user) are not within a normal range (relative to a common table of measurements) or because the user has provided input for this purpose (for example by using a specific menu with a list of the most frequent abnormalities, which can be displayed to the user).

[0111] Identification of abnormal features of the region of interest may include performing organ detection on the ultrasound image.

[0112] For example, in the case of fetal screening, the acquired ultrasound image may contain the head of the fetus, which may be considered part of the region of interest. To facilitate automatic measurement results of various typical measurements taken on the fetal head, the image may undergo organ segmentation, and more particularly skull segmentation.

[0113] Segmentation of the skull may be performed, for example, using a dedicated shape transform, which is tailored to detect elliptical shapes in a robust and accurate manner.

[0114] In another example, identification of abnormal features of the region of interest may include performing mid-sagittal brain plane detection on an ultrasound image of the fetal head.

[0115] The midsagittal plane bisects the head, providing a side profile cross section of the fetal head. In the example of fetal head screening, the midsagittal plane would show a cross section of the fetal head for use in identifying abnormal features.

[0116] The midsagittal plane can be analyzed in a variety of ways. For example, midsagittal plane detection can include performing multi-resolution plate structure detection. Multi-resolution plate structure detection takes the form of multi-resolution analysis that uses eigenvalues ​​of a local Hessian matrix within an image to detect plate structure.

[0117] Additionally, midsagittal plane detection may include performing Hough detection via Hough transform, which is a feature extraction technique for identifying shapes in digital images. For example, Hough transform may be used in the method to identify facial shapes, such as: nose; eyes; skull; etc.

[0118] In addition, the Hough transform can be applied to the results of multi-resolution plate structure detection to select the best plane when rotating around the axis of the skull segmentation result.

[0119] Additionally, multi-resolution plate structure detection and Hough detection may be incorporated into a single dedicated filter that may be applied to ultrasound images during the abnormal feature identification process.

[0120] The accuracy of the midsagittal brain plane can be verified using a deep learning classifier that checks whether certain expected anatomical structures are present in the image (e.g., corpus callosum, nasal bones) or outside the plane (eyes, lateral ventricles).

[0121] Identification of abnormal features may also include extracting the fetal outline and obtaining the nasal chin and / or lower facial angles, examples of which are described below with reference to Figure 4 Discussion. The extraction of the fetal outline can be performed automatically using a learning algorithm.

[0122] Following the identification of abnormal features in step 120, in step 130, one or more available supplemental ultrasound images of interest are determined. Figure 3 , the steps of determining one or more available supplemental ultrasound images are discussed in more detail below.

[0123] The supplemental view may be automatically selected based on known additional views of interest relative to the detected abnormal features. In step 140, the user may be presented with a list of supplemental views based on the abnormal features from which they may manually select a desired view. In other words, one or more available supplemental ultrasound images may be displayed to the user. The user may then select the view that they wish to view, and the selected one or more views may then be displayed for viewing by the user.

[0124] In other words, it is possible to automatically determine the available supplementary views of interest with respect to the detected abnormality. These views can be selected from a collection of previously stored and annotated views of the same patient in the ultrasound system. The user can then decide which views will be displayed from the list of automatically selected views.

[0125] In step 150 , one or more supplemental ultrasound views selected by the user may be displayed.

[0126] Figure 3 A method 200 for determining one or more supplemental ultrasound images is shown.

[0127] The method begins in step 120 where abnormal features are identified, as described above.

[0128] In step 210 , it is determined whether a 3D ultrasound image is available.

[0129] If a 3D ultrasound image is not available, the method proceeds to step 220 where a list of possible supplemental images based on the identified abnormality features is presented. For example, the views may include: a supplemental brain view; facial rendering; urogenital tract view; cardiac view; and finger view.

[0130] The list of these interesting views may be based solely on knowledge of the abnormal features and conditions associated with the features detected at step 120, without knowing whether these views are available. This list is internal and not displayed to the user.

[0131] The list of possible supplemental images may be compared to previously acquired ultrasound images of the subject in step 230. If the previously acquired ultrasound image matches the view of interest acquired in step 220, then it is determined that the previously acquired ultrasound image may be used as a usable supplemental ultrasound image.

[0132] If there is no match between the view of interest and a previously acquired ultrasound image, the view of interest is determined to be unusable as a supplementary ultrasound image. In this case, it is possible that the user may be guided to select an alternative ultrasound image that includes the view, if possible.

[0133] Returning to step 210 , if a 3D ultrasound image is available, the method proceeds to step 250 .

[0134] In case of 3D ultrasound images, it is possible to extract one or more supplementary 2D views from the 3D volume by selecting desired imaging planes within the volume.For the example of fetal screening, the spine of the fetus can be used as a reference structure to extract the supplementary 2D views.

[0135] Therefore, available supplementary images are imaging planes that can be generated from the 3D ultrasound images.

[0136] In step 250, anatomical reference structures are extracted.

[0137] The anatomical reference structure, which may be, for example, the cervical region (neck) of the fetus, may be extracted by, for example, intensity analysis around the fetal skull. In the case where the fetal neck is used as an anatomical reference structure, it may be identified for extraction due to the difference in intensity distribution in the neck region compared to other regions around the skull.

[0138] In step 260, a supplementary 2D view set of one or more supplementary 2D views may then be obtained by using the anatomical reference structures as reference points. In the example of an anatomical region comprising the cervical spine region, a first supplementary view set may contain a 2D ultrasound image of the lower head, including the jaw and oral cavity.

[0139] In step 140 , the method may then proceed to displaying one or more available supplemental ultrasound images formed by the set of supplemental 2D views, as described above.

[0140] The method may further comprise step 270, wherein structural features of the anatomical reference structure are extracted. This may be achieved by automatic extraction of features using a deep learning algorithm in a stack of slices or 2D ultrasound images surrounding the anatomical structure. In the case of extraction of the spine as a structural feature, the spine may be extracted using a 2D ultrasound image in the cervical spine region, an example of which is described below with reference to Figure 5 discuss.

[0141] In step 280 , it may then be determined whether the anatomical reference structure can be tracked in the 3D ultrasound image based on the structural features.

[0142] By way of example, it may be determined whether the 3D ultrasound image volume contains a larger spinal region than the cervical spine region. In other words, it is tested whether the 3D ultrasound image volume contains a spinal column extending beyond the cervical spine region (neck).

[0143] If the 3D ultrasound image does not contain a large spinal region, the method may simply proceed to display one or more available supplemental ultrasound images in step 140 .

[0144] However, if it is determined that the 3D ultrasound image contains a larger spinal region, the tangent of the spinal stack or the spinal tangent can be estimated and used as a structural feature. Supplementary 2D planes of interest can then be obtained based on the larger spinal region (acting as an anatomical reference structure) and the spinal tangent (acting as a structural feature) to form portions of one or more available supplementary ultrasound images. Such views can include slices of the fetal torso and / or abdomen, depending on how much of the fetal spine is contained within the 3D ultrasound image. In other words, if anatomical reference structures can be tracked in the 3D ultrasound image, 2D planes of interest can be extracted from the 3D ultrasound image by using the anatomical reference structures as anatomical landmarks.

[0145] Additionally, anatomical reference structures may include one or more of: organs; and skeletal structures.

[0146] For example, a 3D ultrasound image volume may contain a cardiac region.

[0147] If the heart is present in the detection plane, its presence can be tested using an automated classifier trained on a set of chest planes that do or do not contain the heart, and the plane can be extracted as a cardiac view and stored in a list of available supplementary views of interest. In this case, the heart can serve as the anatomical structure, and the ventricles or atria of the heart can serve as structural features for extracting 2D planes of interest in the cardiac region.

[0148] The view of the heart may form part of a supplementary 2D view group of one or more supplementary 2D views.

[0149] In another example, the 3D ultrasound image volume may include a kidney region.

[0150] If a kidney is present in the detection plane, its presence can be tested using an automated classifier trained on a set of ventral planes that do or do not contain the kidney, and the plane can be extracted as a urogenital view and stored in a list of available supplementary views of interest. In this case, the kidney can serve as the anatomical structure, and the renal artery or renal vein can serve as the structural feature to extract the 2D plane of interest in the kidney region.

[0151] The view of the kidney may form part of a supplementary 2D view group of one or more supplementary 2D views.

[0152] It should be noted that these cardiac and urogenital views are given as examples and similar steps can be applied to other parts of the anatomical structure, depending on the view of interest given the abnormal characteristics and associated pathology. For example, the bladder can also be used as a structure for obtaining a supplementary set of views. In addition, the anatomical structure can include: heart; kidney; bladder; stomach; skull; spine; thorax; femur; humerus; or any other suitable organ or bone structure.

[0153] Each of the images contributing to the set of 2D supplementary views may form part of one or more supplementary 2D views displayed in step 140 .

[0154] The method described above can be adapted to other detected abnormal features, wherein the craniofacial starting point described above is given only as an illustration. For example, if the first detected abnormality is a cardiac abnormality, the same workflow can be used by progressively selecting supplementary views of interest along the spine in two directions (towards the head and towards the pelvis). Similarly, starting from an abdominal abnormality, a supplementary view of interest can be selected in the direction of the head.

[0155] In other words, a complete list of supplementary views can be identified based on the abnormality characteristics, which corresponds to all known views for evaluating a given abnormality. Available supplementary views can then be determined from the list based on the content of the acquired ultrasound image. The available views can then be presented to the user for selection or automatically selected. Alternatively, if additional imaging is required, the user can be guided to obtain one or more of the supplementary views.

[0156] Figure 4 A mid-sagittal plane view 400 of a fetus is shown.

[0157] From this view it is possible to measure both the naso-mental angle α and the lower face angle β. These angles can be measured manually by the user, or the angles can be measured automatically by automatic arrangement of measurement lines relative to the contour.

[0158] If one or both of the (one or more) angles are not within the normal predetermined range, the detection and identification of abnormal features can be triggered. For example, according to "An objective measurement to diagnose micrognathia on prenatal ultrasound." (Palit, G. and Jacquemyn Y. and Kerremans M. Clin Exp Obstet Gynecol 35 (2): 121-123, 2008), an angle lower than 142° for the nasal chin angle α can be considered abnormal. As another example, according to "The fetal mandible: a 2D and 3D sonographic approach to the diagnosis of retrognathia and micrognathia." (Rotten D., Levaillant JM, Marthinez, H. Ducou le Pointe, H. and Vicaut, E. Ultrasound Obstet Gynecol, 19: 122–130 (2002)), an angle lower than 49.2° for the lower facial angle can be considered abnormal.

[0159] Figure 5 Several images 500 of 2D views extracted orthogonal to the spine of the fetus are shown.

[0160] For clarity, the spine has been highlighted by a white circle on the image. It is clear from these images that the fetal spine is the dominant feature of the fetal ultrasound image that can be used as a reference structure for navigating the 3D ultrasound volume. In other words, the spine can be used as a structural feature for navigating the 3D ultrasound image volume along the anatomical reference structure of the spine.

[0161] Figure 6 A schematic representation 600 of a stack 610 of ultrasound images aligned using a fetal spine 620 as a reference is shown.

[0162] As described above, the stack of ultrasound images 610 may be extracted starting at the fetal head 630 and proceeding along the spine until the limits of the initially acquired 3D ultrasound image are reached. If the entire fetus is contained within the 3D ultrasound image volume, the stack of ultrasound images may include a cardiac view and a renal view.

[0163] Additionally, the stack of ultrasound images may be extracted from a starting point other than the fetal head 630 .

[0164] The examples described above have been explained in the context of fetal screening; however, it will be appreciated that the described methods may also be applicable to any other medical ultrasound imaging procedure where obtaining supplementary views may be required.

[0165] By studying the drawings, the disclosure and the appended claims, those skilled in the art can understand and implement other variations of the disclosed embodiments when practicing the claimed invention. In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality. Although specific measures are recited in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. A computer-implemented ultrasound imaging method (100), the method comprising: obtaining (110) an ultrasound image including a region of interest of the object; analyzing one or more features of the ultrasound image; identifying (120) abnormal features of the region of interest based on the analysis of the one or more features; automatically generating (220) a list of possible supplemental ultrasound images of the subject based on the identified abnormal features; automatically determining (130) one or more available supplemental ultrasound images based on the list; and The one or more available supplemental ultrasound images are displayed (140) to a user.

2. The method according to claim 1, wherein: The ultrasound image comprises a 3D ultrasound image, and wherein the determining of the one or more available supplemental ultrasound images comprises obtaining one or more supplemental 2D views from the 3D ultrasound image.

3. The method according to any one of claims 1 to 2, wherein: The identifying (120) of the abnormal features of the region of interest includes performing organ detection.

4. The method according to any one of claims 1 to 2, wherein: The region of interest includes a portion of a fetus.

5. The method according to claim 4, wherein: The identifying (120) of the abnormal features of the region of interest includes performing skeletal and mid-sagittal brain plane detection on the fetus.

6. The method according to claim 5, wherein: The midsagittal brain plane detection includes: Perform multiresolution analysis; and Perform Hough detection.

7. The method according to claim 4, wherein: The identifying (120) of the abnormal features of the region of interest includes: extracting a contour of the fetus; and Based on the contour of the fetus, obtaining: Nasochinal angle (α); or Lower facial angle (β).

8. The method according to any one of claims 1 to 2, wherein: The determining of the one or more available supplemental ultrasound images further comprises: comparing the list of possible supplemental ultrasound images to previously acquired ultrasound images of the subject (230); and In case a previously acquired ultrasound image matches a possible supplementary ultrasound image, the previously acquired ultrasound image is selected as an available supplementary ultrasound image.

9. The method according to claim 2, wherein: The obtaining of the one or more available supplementary ultrasound images from the 3D ultrasound image further comprises: extracting (250) anatomical reference structures; A supplementary set of 2D views is obtained (260) by extracting one or more 2D planes of interest from the 3D ultrasound image based on the extracted anatomical reference structure.

10. The method according to claim 9, wherein: The obtaining of the one or more available supplemental ultrasound images from the 3D ultrasound image comprises: extracting (270) structural features from the anatomical reference structure; determining (280) whether the anatomical reference structure can be tracked in the 3D ultrasound image based on the structural feature; In case the anatomical reference structure can be tracked in the 3D ultrasound image: A 2D plane of interest is extracted from the 3D ultrasound image by using the anatomical reference structure as an anatomical landmark.

11. The method according to claim 10, wherein: The anatomical reference structures include one or more of the following: organs; and Bone structure.

12. The method according to claim 11, wherein: The organs include one or more of the following: heart; kidney; bladder; stomach; and wherein the skeletal structure includes one or more of the following: skull; spine; thorax; femur; and humerus.

13. A computer program product comprising computer program code means adapted to implement the method according to any one of claims 1 to 12 when run on a computer.

14. An ultrasound imaging system (2), the system comprising: an ultrasound probe (4) adapted to obtain an ultrasound image of a region of interest containing the object; as well as A processor adapted to: analyzing one or more features of the ultrasound image; identifying abnormal features of the region of interest based on the analysis of the one or more features; automatically generating a list of possible supplemental ultrasound images of the subject based on the identified abnormal features; One or more available supplemental ultrasound images are automatically determined based on the list; and the one or more available supplemental ultrasound images are displayed to a user.

Images