Image acquisition method

Abstract

The received 3D image frames are processed in real time to extract target planes, and each extracted plane is processed by a quality check algorithm. Records of all target planes acquired with sufficient quality are maintained, for example, by removing them from the list of planes to be captured. Optionally, each new frame may be processed only to extract planes that have not yet been acquired with sufficient quality. The user interface display is dynamically controlled so that the user is always presented with a real-time display of at least one plane among those that still need to be captured with sufficient quality. Thus, the user is constantly provided with optimal guidance images to control the acquisition device and acquire new 3D frames with image data in which the still required 2D planes can be successfully extracted with high quality.

Description

Technical Field

[0001] The present invention relates to the field of medical image acquisition, and more particularly, to a method for acquiring user-guided images.

Background Art

[0002] In the context of medical imaging, it is often necessary to obtain a standardized set of image views of a particular anatomical structure. For example, a standard echocardiography protocol includes the acquisition of the following views: parasternal long axis, parasternal short axis, apical four chamber, subcostal (subxiphoid), and inferior vena cava views. Fetal ultrasound images also involve the acquisition of a series of views to capture standard fetal measurements, such as biparietal diameter (BPD), head circumference (HC), abdominal circumference (AC), and femur length (FL).

[0003] One approach to doing this is to acquire one or more 3D images and extract the necessary planes from the 3D images. This has the advantage that the user does not need to accurately position the ultrasound imaging probe for each required planar view. Instead, a more limited set of one or more 3D images can be acquired, and the required view planes can be extracted from the 3D image data as 2D images.

[0004] There are methods for automatically extracting a set of one or more 2D planes from 3D ultrasound images.

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the context of such a system, it is important that all planes are recorded with sufficient quality. Also, it is preferable to evaluate the image quality in real time during the scan so that additional scan data can be acquired if the image quality is poor. This avoids the situation where sub-optimal image quality becomes apparent only after the patient has been discharged.

[0006] In some cases, a single 3D image may allow all 2D planes to be extracted with sufficient quality. However, often some planes extracted from a given 3D image will have adequate quality, while others will not. For example, some areas of the acquired 3D image may have poor contrast compared to other areas, meaning that only a subset of planes can be extracted with high quality. Therefore, current practice requires the user to manually re-examine the quality of all planes sliced ​​from the current 3D volume during acquisition while viewing it in real time. However, when a large number of planes are involved, the user needs to monitor multiple views together during the scan for the purpose of optimizing image quality at once. This is extremely difficult, as in practice the user can only focus on one view at a time.

[0007] It would be beneficial to provide an ultrasound acquisition method that can mitigate one or more of the above-mentioned problems. [Means for solving the problem]

[0008] This invention is defined by the claims.

[0009] According to one aspect of the present invention, a computer implementation method is provided for use during an ultrasonic scan performed by a user using an ultrasonic acquisition device.

[0010] The method includes the steps of: receiving real-time 3D ultrasound data of an anatomical object from an ultrasound acquisition device, wherein each 3D frame includes a sequence of one or more 3D frames spanning a 3D volume; determining a plurality of target planes that pass through the 3D volume to be extracted; and recording the target planes in a target plane list. The method further includes the step of performing a plane capture operation, which comprises: extracting a selected set of a plurality of target planes determined from the 3D ultrasound data for each of at least one subset of 3D frames of the 3D ultrasound data; controlling a user interface to provide a real-time ultrasound image display output that displays a selected subset (e.g., one) of the extracted set of planes for the current frame; and storing the extracted planes in a plane dataset. The method further includes a step of performing a quality evaluation operation in parallel with the plane capture operation, the quality evaluation operation including the steps of applying a predetermined quality evaluation algorithm to planes in a plane dataset to determine, for each target plane in the target plane list, whether any individual extracted planes in the dataset corresponding to the target plane meet a predetermined quality standard, or whether the time series of extracted planes in the dataset corresponding to the same target plane meets a predetermined quality standard, and updating the target plane list to remove any target planes that meet the predetermined quality standard from the list. The method further includes repeating the steps of the plane capture operation for each subsequent 3D frame until the target plane list is empty. For each frame, the selected set of target planes to be extracted includes at least each of the planes still included in the list. For each frame, the selected subset of extracted planes to be displayed on the user interface includes at least one of the planes still included in the list.

[0011] Embodiments of the present invention provide an image acquisition control sequence in which each relevant frame of a 3D ultrasound data stream is processed in real time to extract a set of target anatomical planes and undergo quality evaluation to check that each of these planes meets quality criteria. Furthermore, it is proposed to maintain a list in local memory that records the set of target planes that have not yet met the defined quality criteria. The list starts with all target planes to be captured included on it and can be updated whenever a plane that meets the quality criteria is newly captured. When the list is empty, all target planes have been captured with sufficient quality.

[0012] The quality assessment may be configured to be applied to a single plane or to a time series of planes. In some embodiments, a particular target plane may be determined to be successfully acquired if at least one single extracted plane corresponding to the target plane is acquired with quality that meets a quality standard. In some embodiments, a particular target plane may be determined to be successfully acquired if the extracted planes of a time series (over a series of frames) corresponding to the target plane meet a predefined quality standard. For example, in the case of cardiac imaging, the time series may span a complete cardiac cycle.

[0013] In each new 3D frame, target planes that are still included in the target plane list, in some embodiments, need to be extracted from the newly acquired volume. This saves processing resources and time. One advantage provided by the proposed method is that it automatically tracks target planes that have not yet been captured with sufficient quality, and automatically performs the extraction, quality check, and storage steps to ensure that a complete set of target planes is acquired with high quality during the scan. A further advantage of the proposed method is that throughout the scan, a real-time display window is provided that shows a subset (preferably just one) of the extracted planes for the latest 3D frame, and the display window is continuously updated so that it always shows planes that have not yet been acquired with sufficient quality (quality that meets the required quality criteria). In this way, the user performing the scan is always presented with a live image of the target view that still needs to be acquired. Thus, the user can focus on adjusting the probe placement to optimize the quality of these target planes.

[0014] Therefore, the proposed method can be understood as two processes operating in parallel, one comprising an automated plane extraction method that operates to automatically extract target planes from a series of 3D frames so that a complete set of target planes is obtained with sufficient quality, and the second presenting the user with a display panel on a user interface display that is continuously updated to show live ultrasound images from at least one view in which the target plane has not yet been successfully acquired.

[0015] Note that the planar capture operation does not need to be performed for every frame received from the ultrasonic acquisition device, but can be performed every x frames, where x is greater than 1. This may be implemented, for example, to save processing resources.

[0016] The above-mentioned criterion subset of extracted planes displayed on the user interface may preferably consist of only a single plane. This can be referred to herein as a “supervised plane” because it is intended to provide the user with a real-time imaging view that can be used, for example, to adjust the positioning of the ultrasound imaging device or to adjust acquisition parameters.

[0017] This method includes the step of determining a set of target planes to be acquired. In some embodiments, this can be achieved simply by accessing a database or data list that records the planes to be acquired. In some embodiments, it may include identifying the planes through anatomical segmentation.

[0018] For example, in some embodiments, the step of determining a plurality of target planes includes: applying an anatomical segmentation algorithm to at least one of the 3D frames to identify one or more predetermined anatomical locations within the 3D frame; searching a predetermined list of target views, each of which is associated with an anatomical location; and determining target planes based on the target views and the segmentation of the 3D frame. Thus, the method extracts planes to obtain target views. Segmentation may be performed for each 3D frame being processed, or only once, and the same target plane may be used for subsequent frames.

[0019] In some embodiments, the method includes displaying a visual representation on the user interface of the target planes remaining on the target plane list. This helps guide the user when performing a scan, as it allows the ultrasound probe to be positioned in a way that optimizes capturing all target planes that have not yet been captured. The visual representation may include, for example, a text list. The visual representation may include a graphical representation of the target planes. In some embodiments, the display may include a display area in which the complete set of extracted planes is displayed as images in tiles, each tile being annotated to indicate whether the associated plane meets or does not meet quality standards.

[0020] In some embodiments, the user interface includes a first display area and a second display area, and the method includes controlling the user interface to display a selected subset (e.g., one) of a plurality of extracted planes having a first image size in the first display area, and controlling the user interface to display a plurality of the extracted planes, e.g., all of them, in the second display area, each having an image size smaller than the first image size. For example, the second display area may display the extracted planes as an array of smaller tiles, while the first display area shows a larger image to guide the user when actively positioning the ultrasound probe.

[0021] In some embodiments, the method includes controlling the user interface to display all extracted planes of the current frame in a second display area, with each plane annotated with a visual indication of whether it remains on the target plane list. For example, the visual indication may be a boundary around the displayed target plane, and the graphical or visual properties of the boundary are changed depending on whether the plane is still on the list. For example, the graphical or visual properties may be the color of the boundary.

[0022] In some embodiments, the quality classification can include a graded quality score. In some embodiments, the quality classification can include a binary pass / fail classification.

[0023] In some embodiments, the method includes providing user control that controls a user interface to enable the user to invalidate the quality classification output from the quality assessment for each extracted plane, thereby enabling the user to manually remove any extracted plane from the list of target planes to be extracted or to manually re-add any extracted plane to the list of target planes to be extracted.

[0024] In other words, it is proposed to control the user interface to provide a user control option that enables the user to manually set the quality classification of a particular plane or series of planes to pass (or fail).

[0025] In some embodiments, the quality assessment algorithm can be configured to identify the extracted planes that pass or fail the quality standard by only a margin less than a predefined threshold margin and to classify the extracted plane or series of planes using a boundary classification. The method can comprise controlling the user interface to display a visual representation of the target plane using the boundary classification and to prompt the user for user input. The quality classification can be fixed as pass or fail depending on the user input.

[0026] In other words, according to this series of embodiments, for an image having a boundary quality score (e.g., close to an acceptable threshold), it is proposed that the user be asked whether the user considers the quality to be sufficient.

[0027] The present invention can also be implemented in the form of software. Therefore, another aspect of the present invention is a computer program product comprising computer program code configured to cause a processor, when executed on the processor, to execute a method according to any of the embodiments or examples described herein or according to any of the claims of the present application.

[0028] The present invention can also be implemented in hardware form. Therefore, another aspect of the present invention is a processing device for use with an ultrasonic acquisition device during an ultrasonic scan performed by a user, the processing device comprising an input / output and one or more processors. The one or more processors are configured to receive real-time 3D ultrasonic data from an ultrasonic collection device of an anatomical object, each 3D frame including a series of 3D frames spanning a 3D volume, determine a plurality of target planes through the 3D volume, extract the target planes into a target plane list, and record them.

[0029] The one or more processors are further configured to perform a plane capture operation. The plane capture operation includes, for each of at least one subset of the received frames of the 3D ultrasonic data, extracting a selected set of the plurality of target planes determined from the 3D ultrasonic data, controlling a user interface to provide a real-time ultrasonic image display output for displaying a selected subset (e.g., one) of the extracted planes of the current frame, and storing the extracted planes in a plane data set.

[0030] One or more processors are further adapted to perform a quality evaluation operation in parallel with the plane capture operation, which includes applying a quality evaluation algorithm to planes in the plane dataset to determine, for each target plane in the target plane list, whether the individual extracted planes in the dataset corresponding to the target plane meet a predefined quality standard, or whether the time series of extracted planes in the dataset corresponding to the same target plane meets a predefined quality standard, and updating the target plane list to remove any target planes that meet the predefined quality standard from the list.

[0031] This method further includes repeating the plane capture operation step for each subsequent 3D frame in the sequence of frames until the target plane list is empty.

[0032] The quality evaluation operation may be repeated at intervals in parallel with the planar capture operation. In some examples, this may be performed every time the planar capture operation is executed.

[0033] For each frame, the selected set of multiple target planes to be extracted includes at least each of the planes that are still included in the list. For each frame, the selected subset (e.g., one) of the multiple extracted planes to be displayed on the user interface includes at least one of the planes that are still included in the list.

[0034] Another aspect of the present invention is a system comprising a processing apparatus according to any embodiment or example described in this document or the claims of this application, and an ultrasonic acquisition system equipped with an ultrasonic probe for user operation. In some embodiments, the system may further include a user interface equipped with a display unit.

[0035] These and other aspects of the present invention are evident from the embodiments described below and will be explained with reference thereto.

[0036] To better understand the present invention and to more clearly illustrate how it can be implemented, the accompanying drawings are referenced here, merely as examples. [Brief explanation of the drawing]

[0037] [Figure 1] The block diagram outlines the steps of an exemplary method according to one or more embodiments of the present invention. [Figure 2] This is a block diagram showing exemplary components of an apparatus and system according to one or more embodiments of the present invention. [Figure 3] This shows an exemplary display screen of a user interface according to one or more embodiments of the present invention. [Figure 4] This shows the changes in the generated display screen of an exemplary user interface after each new sequence of 3D image frames has been acquired. [Figure 5] An exemplary ultrasound acquisition device is shown. [Modes for carrying out the invention]

[0038] The present invention will be described with reference to the figures.

[0039] Detailed descriptions and specific examples illustrate exemplary embodiments of the apparatus, system, and method, but should be understood to be illustrative only and not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, system, and method of the invention will be better understood from the following description, the appended claims, and the appended drawings. Please understand that the figures are for illustrative purposes only and are not drawn to scale. Also, please understand that the same reference numerals are used throughout the figures to indicate the same or similar parts.

[0040] The present invention provides a method for obtaining a set of target plane views through an anatomical structure of interest via processing of 3D ultrasound image data. The method processes received 3D image frames in real time to extract target planes, and processes the extracted planes individually or as a time series using a quality check algorithm. Records of all target planes acquired with sufficient quality are maintained, for example, by removing them from the list of planes to be captured. Optionally, each new frame may be processed only to extract target planes that have not yet been acquired with sufficient quality. A user interface display is dynamically controlled to always present the user with a real-time display of at least one plane view of which still needs to be captured with sufficient quality. Thus, the user is constantly provided with optimal guidance images for controlling the acquisition device to acquire new 3D frames containing image data that can successfully extract the still required 2D planes with high quality.

[0041] In other words, it is proposed to provide an acquisition system and method that ensures that for every desired 2D plane, at least one 3D image is recorded in which the 2D plane can be extracted (sliced) with sufficient quality, or that at least one 3D frame of a predefined time span (e.g., one complete cardiac cycle, or one complete phase of a cardiac cycle) is recorded in which a time-series frame (e.g., cine clip) of the associated 2D plane can be extracted (sliced) with sufficient quality.

[0042] According to one particular set of exemplary embodiments, this can be achieved by applying a monitoring protocol as follows:

[0043] The user can acquire a first 3D ultrasound volume while monitoring a reduced set of sliced ​​2D planes (e.g., only a single plane). Here, "monitoring" means that a real-time display image of the reduced set of 2D planes is presented on the user interface display, which the user can use to guide image acquisition. The plane extraction algorithm can extract (slice) all desired 2D planes from the 3D volume. The automatic quality detection algorithm can analyze the extracted planes to determine which of the sliced ​​2D planes have sufficient quality and which do not. Since the user is only presented with a reduced set of target planes, the number of planes that do not meet the quality criteria may exceed the number of planes the user is monitoring. If all sliced ​​planes have sufficient quality, acquisition can be completed and the method can be terminated. Otherwise, a second 3D ultrasound volume is acquired, while the user interface is updated to display a subset of previously extracted 2D planes that did not have sufficient quality when evaluated by the quality detection algorithm. In other words, the monitoring view is dynamically adjusted to one or more of the target 2D planes that do not yet meet the quality criteria. This procedure is repeated from at least one of the acquired 3D frames, preferably from a time series of 3D frames over a predefined time span, such as the complete motion cycle of an imaged anatomical object, or the complete cardiac cycle, until all target planes can be extracted with sufficient quality (i.e., further 3D volumes are acquired).

[0044] The steps of an exemplary method according to one or more embodiments of the present invention are more explicitly outlined in block diagram form in Figure 1. The steps are first described in summary form, followed by a more extended description of the key features.

[0045] Method 10 shown in Figure 1 is intended as a computer implementation method for use during an ultrasound scan performed by a user using an ultrasound acquisition device. This method includes receiving 3D ultrasound data of an anatomical object from the ultrasound acquisition device (12). This may be real-time 3D ultrasound data, i.e., live ultrasound data, i.e., a stream of ultrasound data. The 3D ultrasound data may include a sequence of one or more 3D image frames, each 3D frame spanning a 3D volume.

[0046] This method further includes determining a number of target planes through a 3D volume to be extracted 14, and recording the target planes in a target plane list 16. The target plane list may be recorded in local short-term memory. The number of target planes may be determined by a lookup from a list or database, or by a computation process as described later.

[0047] Method 10 further includes performing a plane capture operation 20 for each of at least one subset of 3D ultrasound planes. The plane capture operation may be performed for each frame of ultrasound data, or for a subset of frames, e.g., every x frames, where x is greater than 1. It may be performed for selected frames and may be simply repeatedly re-triggered each time it finishes, for example, as long as the target plane list 16 remains non-empty, and applied each time to the most recent 3D ultrasound frame received. It may also be performed for a user-selected frame in response to user input, for example, from trigger control, and the operation is performed for the most recently received 3D frame.

[0048] The planar capture operation 20 is performed for each 3D frame to which it is applied. Step 22 involves extracting a selected set of multiple target planes determined from 3D ultrasound data, wherein for each frame, the selected set of target planes to be extracted includes at least each of the planes still included in List 16. Step 28: A step of controlling a user interface to provide a real-time ultrasound image display output that shows a selected subset of the extracted set of planes for the current frame, wherein for each frame, the selected subset of extracted planes displayed on the user interface still includes at least one of the planes included in List 16. The steps include storing the extracted planes in the plane dataset 17, A step to check whether the target plane list 16 is empty, and if the target plane list 16 is not empty, a step 30 to repeat the step of the plane capture operation 20 for the next 3D frame in the frame sequence. It holds.

[0049] In parallel with the planar capture operation 20, this method Step 24: For each target plane in the target plane list, apply a predetermined quality evaluation algorithm to the planes in the plane dataset 17 to determine whether any individual extracted planes in the dataset corresponding to the target plane meet a predetermined quality standard, or whether the time series of extracted planes in the dataset corresponding to the same target plane meets a predetermined quality standard. Step 26 updates the target plane list and removes any target planes that meet predetermined quality standards from the list. The process also includes a step of performing a quality evaluation operation 21 which has the following characteristics:

[0050] The steps of the plane capture operation 20 may be repeated for each subsequent 3D frame in the sequence of frames until the target plane list is empty, and the quality evaluation operation 21 may be repeatedly performed in a loop, for example, until the target plane list 16 is empty.

[0051] The quality evaluation operation 21 may be executed in parallel with the plane extraction procedure, but at intervals. For convenience, it may be triggered each time the plane extraction operation is triggered, for example, each time it is executed immediately before or after each iteration of the plane extraction procedure. Alternatively, it may be executed completely independently of the plane extraction operation, at different intervals. The results are supplied to the plane extraction operation by updating the target plane list, which is the output operation of the quality evaluation operation.

[0052] As is clear from the above overview, the objective of the quality evaluation operation 21 is to review the planes extracted from the sequence of 3D frames and check that, for each target plane, a defined quality standard for that target plane is met. The plane extraction operation 20 can operate in a loop over the received sequence of 3D frames, with each time it extracts a set of target planes from the 3D data, records them in the plane dataset 17, and records them along with a timestamp corresponding to the time of acquisition. This plane capture operation may proceed until the target plane list is empty. Simultaneously and in parallel, the quality evaluation operation 21 can periodically or continuously review the extracted plane dataset 17 compiled so far and check whether, for each target plane still listed in the target plane list 16, a predefined quality standard / criteria is met by the set of planes extracted in the plane dataset. In some cases, this may include checking, for each target plane, whether at least one of the extracted planes meets a predefined quality standard. In other embodiments, it may include checking, for each target plane, whether a specific time sequence of frames of the target plane that meets a particular quality standard has been acquired. Planes or sequences of planes that meet quality standards can be tagged or annotated for easy identification later. In some embodiments, the final plane dataset may be compiled as a further step, from extracted planes in a plane dataset 17, each containing a single instance of the target plane (or time series), which has been tagged as meeting quality standards.

[0053] In some embodiments, the determination of whether a target plane has been satisfactorily obtained can be evaluated at the level of a complete cardiac cycle. Thus, the quality assessment 24 for each target plane includes determining whether the sliced ​​time series of the target frame over a complete cardiac cycle conforms to a quality criterion.

[0054] Therefore, the aforementioned quality evaluation algorithm may be configured to receive a single frame as input, or in other embodiments, it may receive a planar time series as input.

[0055] The user interface may include a display. In some embodiments, a first subregion of the display is used to display the aforementioned subset of extracted planes (sometimes referred to herein as “supervised planes”) at high resolution (e.g., as a large image). In some embodiments, a second subregion of the display is used to display a larger set (e.g., all) of extracted planes at a lower resolution (e.g., as a smaller image). In some embodiments, the smaller images of the planes may be further annotated with a visual status indicator showing whether planes have already been extracted with sufficient quality from one of the acquired 3D volumes.

[0056] As described above, the method can also be carried out in hardware form, for example, in the form of a processing apparatus configured to perform the method according to any example or embodiment described herein, or according to any claim of this application.

[0057] To further aid understanding, Figure 2 presents a schematic diagram of an exemplary processing apparatus 32 configured to perform a method according to one or more embodiments of the present invention. The processing apparatus is shown in the context of a system 30 that includes the processing apparatus. The processing apparatus alone represents an aspect of the present invention. System 30 is another aspect of the present invention. The provided system does not need to include all of the illustrated hardware components, but may include only a subset thereof.

[0058] The processing unit 32 comprises one or more processors 36 configured to perform the methods outlined above or the methods according to any embodiment described herein or in any claim of this application. In the illustrated example, the processing unit further comprises input / output 34 or a communication interface.

[0059] In the example shown in Figure 2, the system 30 further includes a user interface 52. The user interface may include a display.

[0060] The system 30 further comprises an ultrasound acquisition or imaging device 54 for acquiring 3D ultrasound imaging data 44. Details of an example of the ultrasound acquisition device 54 will be described later with reference to Figure 5.

[0061] In some embodiments, the user interface 52 may be integrated with the ultrasonic acquisition device 54. For example, the display of the ultrasonic acquisition device may be used as the display for the user interface 52. In further embodiments, the user interface may have a separate display unit from the ultrasonic acquisition device 54.

[0062] The system 30 may further include a memory 38 for storing computer program code (i.e., computer executable code) configured to cause one or more processors 36 of the processing unit 32 to perform the method outlined above, or any embodiment described herein, or any claim thereof.

[0063] As described above, the present invention can also be implemented in software form. Accordingly, another aspect of the present invention is a computer program product comprising computer program code configured, when executed on a processor, to cause the processor to perform a method according to any example or embodiment of the present invention described herein, or according to any claim of this patent application.

[0064] The ultrasound acquisition device 54 may include an ultrasound transducer unit such as an ultrasound probe having multiple ultrasound transducers, for example, an array transducer. In some embodiments, the probe may be a handheld probe. In some examples, the probe may be a transesophageal (TEE) probe. In some examples, the probe may be a transthoracic (TTE) probe.

[0065] As described above, it is not essential that the system 30 itself includes an ultrasonic acquisition device. Instead, the processing unit 32 may be adapted to be communicatively coupled to the ultrasonic acquisition device during use, for example, via a network link such as a DNL link, or for example, via an internet connection. Therefore, in some embodiments, the ultrasonic acquisition device may be located away from the actual processing unit that implements the method.

[0066] As described above, Method 10 includes the steps of first determining a plurality of target planes through the extracted 3D volume and recording the target planes in a target plane list 16. In other words, this method includes compiling an initial version of the target plane list 16. In some embodiments, determining a plurality of target planes includes accessing a pre-stored list or database that records the intended target planes. In some embodiments, the step of determining a plurality of target planes may include processing one or more of the acquired 3D images with one or more anatomical segmentation algorithms to identify the target planes.

[0067] For example, according to at least one set of exemplary embodiments, the step of determining a plurality of target views may include: searching a predetermined list of target views, each of which is associated with an anatomical location; applying an anatomical segmentation algorithm to at least one of the 3D frames to identify one or more predetermined anatomical locations within the 3D frame; and determining a plurality of target views based on the target views and the segmentation of the 3D frame.

[0068] The application of segmentation establishes an anatomical context from a 3D volume. For example, segmentation can employ the use of model-based segmentation. For at least one preferred exemplary method of model-based segmentation, see the paper: Ecabert, O et al. Automatic Model-Based Segmentation of Heart in CT Images Medical Imaging, IEEE Transactions on, 2008, 27, pp. 1189–1201.

[0069] As a further example, segmentation can utilize machine learning-based segmentation algorithms, such as the use of a convolutional neural network trained to perform segmentation. At least one suitable machine learning model for performing segmentation is described in LI, Yuanwei, et al. Standard plane detection in 3d fetal ultrasound using an iterative transformation network. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2018. S. 392–400.

[0070] Based on the anatomical context provided by segmentation, the method includes determining the location of a defined set of 2D views within a 3D volume. For example, the location of a standard view plane can be encoded into a segmentation model by assigning labels to one or more segmentable anatomical elements located within the view plane. The segmentation result provides a set of segmentable elements labeled with respect to their location in the 3D frame volume. Linear regression can then be performed through the segmented elements to determine the closest fitting plane through the 3D frame volume that coincides with the standard view plane.

[0071] Once the target plane is identified, the plane can be extracted from the 3D frame by slicing the 3D frame. The extracted plane can be stored in a plane dataset 17.

[0072] With respect to the aforementioned predetermined list of target views, these may correspond to a standard set of view planes, for example, according to a standardized imaging protocol. For example, in the field of cardiac imaging, a standard echocardiographic protocol includes the acquisition of the following views: parasternal long axis, parasternal short axis, apical four-ventricle, subxiphoid (subcostal), and inferior vena cava views. Further, simpler examples include protocols that include apical four-ventricle views, apical two-ventricle views, aortic valve views, and mitral valve views, all of which are acquired from a single fixed probe position, e.g., the apical position.

[0073] As a further example, fetal ultrasound imaging involves acquiring a series of views to capture standard fetal measurements, such as greater wall diameter (BPD), head circumference (HC), abdominal circumference (AC), and femur length (FL).

[0074] In some embodiments, the user interface may be controlled to provide the user with control options, allowing the user to select from among several possible sets of target views, each associated with a standard imaging protocol. These may be called presets. Thus, in this example, the user can select from several presets, each resulting in a defined set of 2D view planes to be sliced. In some embodiments, the user interface may be controlled to allow the user to freely define a set of target views, for example, by selecting from a database or a predefined dataset of possible target views.

[0075] As described above, the proposed method involves applying a predetermined quality evaluation algorithm to planes in the plane dataset 17 to determine whether each target plane in the target plane list satisfies a predefined quality criterion. In some embodiments, the quality evaluation algorithm is configured to determine whether any individual extracted planes in the dataset corresponding to each given target plane satisfy a predefined quality standard. In some embodiments, the quality evaluation algorithm is configured to determine, for each target plane in the target plane list, whether the time series of extracted planes in the dataset corresponding to the same target plane (i.e., a series of plane frames) satisfies a predefined quality standard.

[0076] In some embodiments, the quality evaluation algorithm may be configured to receive sliced ​​planes or time-series sliced ​​planes along with metadata indicating which target anatomical view planes they represent (e.g., 2-ventricular view, 4-ventricular view). Image processing allows the quality evaluation algorithm to provide a determination as to whether the sliced ​​planes or time-series of planes have sufficient image quality.

[0077] For example, when quality assessment is applied to a single plane, the quality assessment algorithm may be based on, for instance, the degree of visibility of one or more anatomical structures (e.g., walls, valves, ventricles), the degree of correspondence between the view represented by the plane and the view on the target plane, or an analysis of the quality level of the image's contrast or noise levels.

[0078] When a quality evaluation algorithm is applied to a time series of planes in the same target plane view, the algorithm can check that all planes in the time series are of acceptable quality based on similar factors evaluated for each plane in the time series. For example, each plane in a series may be evaluated to determine the degree of visibility of one or more anatomical structures (e.g., walls, valves, ventricles), the degree of correspondence between the view represented by the plane and the view in the target plane, or the quality level of the contrast or noise level of the image. In other examples, quality criteria specific to the time series, such as the absence of motion artifacts or the spatial alignment of the planes in the series, may be evaluated.

[0079] The output of an image quality evaluation algorithm may, in some embodiments, include a binary yes / no or pass / fail classification or decision indicating whether a plane or set of planes meets quality criteria. Additionally or alternatively, the output may include non-binary grading, such as graded quality scores, to which a threshold may be applied to determine whether a predefined quality criterion is met. If the output score is above the threshold, the quality is sufficient. If the quality is below the threshold, the quality is insufficient.

[0080] In some embodiments, the quality evaluation algorithm may be a machine learning algorithm such as an artificial neural network.

[0081] As an example, a quality assessment algorithm may include a trained convolutional neural network (CNN). The CNN may be trained using a training dataset containing sliced ​​images or time sequences of images, each annotated with a corresponding quality score. The quality score annotations may be based on manually assessed quality determined by human experts.

[0082] A CNN can be trained to provide binary output classification (e.g., pass / fail) or to provide grade classification scores.

[0083] A CNN can be trained differently to generate scores related to different possible features of the input image plane. For example, a CNN could be trained to output scores indicating how well a particular anatomical structure (e.g., wall, valve) is visible in the image, how closely a plane matches a target plane, or the quality of the image's contrast / noise level.

[0084] To evaluate the quality of a plane time series, one option is to apply the aforementioned CNN to each plane in the series, derive a quality classification for each plane, determine whether each plane meets its individual quality criteria, and if all planes constituting the series meet their individual quality criteria, the entire series is judged to meet the quality criteria.

[0085] Another option would be to train a CNN to take a set of planes as input and determine quality classifications based on factors related to the set of planes, such as the proper alignment between the planes.

[0086] In either case, the CNN can be trained using a supervised training procedure to predict a quality score based on the input images. Score prediction can be implemented as a regression or classification problem. A segmentation-based approach is also possible, but may be undesirable due to greater annotation requirements.

[0087] After training, the CNN can be applied to sliced ​​images during a scan session in real time. In some embodiments, if the output of the CNN is a stepped score, this score can be compared to a threshold to determine whether the relevant extracted plane or set of planes meets a predetermined quality standard. Alternatively, if the output of the CNN is a binary pass / fail classification, the algorithm's output directly provides an indication of whether the extracted plane or set of planes meets a quality standard.

[0088] In some embodiments, a CNN can be trained to generate multiple different quality scores, each associated with a different feature of an input image or a series of images (as in the example above). The different scores generated by the CNN can be used directly or merged into a single quality score, for example, by averaging or weighted averaging.

[0089] Part of this method involves controlling a user interface 52 to provide display output to the user, both of which advise the user on which of the target planes have already been acquired with sufficient quality, and provide image acquisition guidance through the display of a real-time image view corresponding to at least one of the target planes / views that has not yet been captured with sufficient quality.

[0090] Figure 3 shows an exemplary display output of the user interface 52 according to at least one set of embodiments.

[0091] Different display areas are provided for the display output.

[0092] The first display area 60 provides a real-time ultrasound image display output that shows a selected subset of the extracted set of planes in the current frame. In this example, the selected subset consists of a single plane from the extracted set of planes in the current frame. In particular, a single plane from the extracted planes that does not yet meet the predefined quality criteria is displayed. That is, the first display area 60 displays the current image plane recorded in the target plane list 16. The first display area displays the subset of extracted planes in higher resolution or larger size compared to the image presented in the second display area 70. The function of the first display area is to guide the operator during a continuous ultrasound scan procedure. The operator can look at the first display area and adjust the positioning of the ultrasound probe to attempt to acquire a new 3D image frame that captures the target plane drawn in the first display area with sufficient quality. Since this method operates in real time, new 3D frames are acquired at a relatively high frequency, e.g., multiple times per second. Therefore, the image drawn in the first display area is continuously updated and always shows the image plane extracted from the latest 3D frame. In this way, the first display area provides a real-time display output to guide image acquisition. In practice, the first display area provides a real-time ultrasound image output, and the particular 2D plan view shown always corresponds to one of the target planes that still remain to be captured with sufficient quality, i.e., one of the target planes still recorded in the target plane list 16. The image planes drawn in the first image area may be called “supervised” planes, allowing the user to easily teach the image quality of these planes during acquisition.

[0093] The display output in the example of Figure 3 further comprises a second display area 70 controlled to display all or at least a subset of the extracted planes at a lower resolution and / or a smaller size. For illustrative purposes, Figure 3 shows an example where there are four target planes to be acquired in total 72a, 72b, 72c, and 72d. Thus, the second display area is controlled to display multiple, for example, all of the extracted planes, each having an image size smaller than the image size of the image shown in the first display area 60.

[0094] Optionally, the display output of the user interface 52 can be further controlled to include one or more additional display areas. For example, as shown in Figure 3, the display output may include a third display area 80 that shows a volume rendering of an anatomical object or feature located within the current 3D frame.

[0095] To help understand the overall process flow of the method, an illustrative method flow is described here.

[0096] At the start of the scan, the target plane list 16 (see Figure 1) is loaded with a record of the full list of target planes to be captured during the scan.

[0097] The user acquires a first 3D ultrasound frame using an ultrasound acquisition device. For the first frame, an image processing algorithm slices all target planes recorded in the target plane list 16 from the frame. The user interface 52 is controlled to display a reduced subset of the extracted planes, for example, a single extracted plane, in the first display area 60. This provides the user with real-time "supervised" planes that they can focus on in order to guide the continuous acquisition of ultrasound image data, i.e., to guide the acquisition of the next 3D frame.

[0098] A quality evaluation algorithm is applied to the extracted 2D planes. In this described embodiment, the quality evaluation is applied to each extracted plane (for a single point in time). As described above, in further embodiments, the quality evaluation can be applied to a temporal sequence of the extracted planes. An exemplary implementation of the latter option will be described further later.

[0099] Continuing with this example, the quality evaluation algorithm analyzes which of the sliced ​​2D planes have sufficient quality (i.e., meet a predefined quality standard) and which do not. If all the planes extracted from the first 3D frame have sufficient quality, the acquisition is complete. This can be communicated to the user via a user interface.

[0100] Otherwise, the target plane list 16 is updated to remove all planes acquired with sufficient quality from the list. For each target plane acquired with sufficient quality, a record may be maintained indicating the specific 3D frame from which the acceptable quality plane was extracted.

[0101] A second 3D ultrasound volume is acquired. This is processed to extract at least each of the planes still recorded in the target plane list (i.e., not yet captured with sufficient quality). The supervised plane shown in the first display area 60 of the user interface is updated to show one of the newly extracted planes still on the target plane list. In this way, the user is constantly monitoring the image view for any planes that have not yet been acquired with sufficient quality. In other words, the monitoring view 60 is dynamically adjusted to one or more of the missing 2D planes.

[0102] This procedure is repeated until the target plane initially loaded into the target plane list 16, or at least one of the acquired 3D volumes, can be sliced ​​with sufficient quality (i.e., further 3D volumes are acquired).

[0103] If a 2D plane can be sliced ​​from two or more of the acquired 3D frames, the results of the quality assessment can be used to select the 3D volume from which the plane can be sliced ​​with the best quality (for example, the volume with the highest quality score for the relevant plane).

[0104] To further guide the user, the method may further include displaying a visual representation of the remaining target planes on the target plane list 16 on the user interface 52.

[0105] For example, a small image of an extracted plane presented in the second display area 70 may include a status indicator to show whether the plane was extracted with sufficient quality. In other words, the method may include controlling the user interface to display all extracted planes of the current frame in the second display area 70, each plane being annotated with a visual indicator of whether it remains on the target plane list. For example, the visual indicator may be a boundary around the displayed target plane, and the graphical or visual properties of the boundary may change depending on whether the plane is still on the list. For example, the graphical or visual properties may be the color of the boundary. In a further example, the visual indicator may be a colored flag or a traffic light (e.g., green, yellow, red) presented adjacent to each image plane in which it is depicted.

[0106] This is schematically illustrated in Figure 4, which shows the updated display output of the user interface 52 for each of three consecutive 3D frames. These frames may be directly time-sequentially acquired by the ultrasonic acquisition device, or, in some embodiments, frames whose acquisition is manually triggered by the user by a user input device, such as a trigger control provided on the ultrasonic probe. For example, the user can manually trigger the acquisition of a new frame after adjusting the positioning of the probe. In some examples, the method involves receiving a real-time series of 3D frames, but the position extraction operation is applied only to a subset of these received planes, and the subset is determined by user input. For example, the user can manipulate the user control when they want the current frame to be passed to the plane extraction operation.

[0107] Figure 4(a) shows the user interface display output for the first acquired 3D frame. The first display area shows one of the extracted image planes, and the second display area 70 shows all of the extracted image planes. At this stage, none of the extracted planes have yet been acquired with a quality that meets the predefined quality criteria. This leads to the acquisition of the next 3D frame. After extracting all target planes from the second 3D frame, it is found that two of the extracted planes meet the predetermined quality criteria, as evaluated by the quality evaluation algorithm. As shown in Figure 4(b), the second display area 70 displays all the extracted planes again, but with boundary highlighting to show the two extracted planes that meet the quality criteria. In the figure in Figure 4(b), this is indicated by a diagonal boundary. The first display area 60 is also updated to show one of the remaining two target planes that have not yet been extracted with sufficient quality. Next, the next 3D frame is acquired, and at least two target planes that remain to be extracted with sufficient quality are extracted. Of these, one more plane is found to meet the quality standard, and therefore, as shown in Figure 4(c), the second display area 70 is updated to add boundary highlighting to the extracted plane that meets the quality standard. The first display area 60 is updated to display the last remaining plane that has not been extracted with sufficient quality. This method continues until all target planes have been extracted with a quality that meets the quality standard.

[0108] In some embodiments, the user interface 52 may be controlled to allow the user to select any of the target planes displayed in the first display area 60. For example, the user may select the plane with the lowest quality score currently. For example, a status indicator applied to each of the extracted planes in the second display area 70 may indicate the quality score value, for example, using different colors or numbers.

[0109] In some embodiments, priority may be defined in relation to target planes in the target plane list 16. This may be included as part of the target plane list initially loaded at the start of the scan. In some embodiments, during the scan, the planes displayed in the first display area 60 may be determined according to priority, with the planes with the highest priority among those still remaining in the target plane list 16 at a given time being displayed in the first display area. In this way, the user is guided to acquire more important planes first, i.e., more important planes are monitored first. In this way, if the scan needs to be stopped before completion (for example, due to the patient's condition), the most important planes have been acquired first.

[0110] As described above, as a variation of the example above, quality assessment for a target plane / view may be applied at the time-series level of extracted planes corresponding to that target plane over a period of time. The period may be defined according to one or more characteristic anatomical events, e.g., the motion cycle of the imaged anatomical object. For example, in the context of cardiac ultrasound imaging, it is common to acquire a 4D imaging dataset containing time-series 3D image frames spanning one or more complete cardiac cycles, or one or more examples of specific phases of a cardiac cycle, e.g., systole or diastole. In some embodiments, the method may comprise acquiring a sequence of 3D frames spanning at least one cardiac cycle and applying the plane extraction operation 20 to each 3D frame in the sequence to acquire, for each frame, a set of target planes still included in the target plane list 16. The same set of target planes is extracted from each frame in the sequence. Subsequently, the quality assessment procedure 21 is applied to the 2D planes extracted from the sequence of 3D frames over the entire period. In this example, the quality assessment algorithm is configured to receive time-series images for each of the target planes. For each target plane, the time series of each extracted plane corresponding to that target plane is processed by a quality evaluation algorithm to determine whether it meets the quality standards. In that case, the target plane list 16 may be updated to remove the respective target plane. Thus, this embodiment differs from the above embodiment only in that the quality evaluation is performed at the level of a series of planes over an extended period, rather than on a single plane at a single point in time. As an example, the quality evaluation operation 20 may be triggered in this case after the 3D frame of each time series has been acquired and after the plane extraction operation 20 has been applied to each time series.

[0111] In some embodiments, the method may include providing user control to enable the user to control the user interface so that the user can override the quality classification output from the quality assessment for each of the extracted planes or sets of planes (if necessary), thereby enabling the user to manually add or remove any extracted planes from the list of target planes to be extracted.

[0112] Therefore, the user is permitted to override one or more decisions of the quality evaluation algorithm, for example, by manually setting the quality pass / fail classification to pass (or vice versa), and, if necessary, update the quality indication in the second display area 70, or the plane or series or associated with the plane, to green, for example. The first display area 60 may then be updated with the display of the next plane among the extracted planes that are still on the target plane list.

[0113] This could be useful, for example, if the user disagrees with one or more of the automatically determined quality classifications of the extracted planes, or if the user prefers a somewhat different definition of the four-ventricular view, but still wants the system to automatically control the quality of the remaining views.

[0114] In some embodiments, the method may include controlling the user interface to generate prompts asking the user whether the quality of one or more of the extracted planes, or a set of planes, is considered sufficient. For example, this may be done for planes where the quality score output from the quality evaluation algorithm is a boundary line (e.g., close to the acceptance threshold). In this case, the user is asked whether they consider the quality to be sufficient.

[0115] For example, in some embodiments, the quality evaluation algorithm is configured to identify extracted planes or sets of planes that pass or fail the quality standard only if their margin is below a predefined threshold margin, and to classify the extracted planes using boundary classification. The method may include controlling the user interface to display a visual representation of the planes using boundary classification and to prompt the user for user input. The quality classification can then be fixed as pass or fail in response to user input.

[0116] As described above, certain embodiments may include an ultrasound acquisition device 54 and / or means for processing ultrasound echo data to derive further data.

[0117] Refer to Figure 5 for a more detailed explanation of the general operation of an exemplary ultrasonic acquisition device.

[0118] The system comprises an array transducer probe 104 having a transducer array 106 for transmitting ultrasound and receiving echo information. The transducer array 106 may comprise CMUT transducers, piezoelectric transducers formed from materials such as PZT or PVDF, or any other suitable transducer technology. In this example, the transducer array 106 is a two-dimensional array of transducers 108 capable of scanning either a two-dimensional plane or a three-dimensional volume of the region of interest. In another example, the transducer array may be a one-dimensional array.

[0119] The transducer array 106 is coupled to a microbeamformer 112 that controls the reception of signals by the transducer elements. The microbeamformer is capable of at least partial beamforming of signals received by a subarray of transducers (commonly referred to as a “group” or “patch”), as described in U.S. Patents No. 5,997,479 (Savord et al.), No. 6,013,032 (Savord), and No. 6,623,432 (Powers et al.).

[0120] It should be noted that the microbeamformer is generally entirely optional. Furthermore, the system includes a transmit / receive (T / R) switch 116, which may couple a microbeamformer 112, switches the array between transmit and receive modes, and protects the main beamformer 120 from high-energy transmit signals when the microbeamformer is not used and the transducer array is operated directly by the main system beamformer. The transmission of the ultrasonic beam from the transducer array 106 is directed by a transducer controller 118 coupled to the microbeamformer by the T / R switch 116, and a main transmit beamformer (not shown) that can receive input from user operation via a user interface or control panel 138. The controller 118 may include transmit circuits configured to drive the transducer elements of array 106 (directly or via the microbeamformer) during transmit mode.

[0121] The functionality of the control panel 138 in this exemplary system can be facilitated by an ultrasonic controller unit according to one embodiment of the present invention.

[0122] In a typical line-by-line imaging sequence, the beamforming system within the probe can operate as follows: During transmission, the beamformer (which may be a microbeamformer or main system beamformer, depending on the implementation) activates the transducer array or sub-apertures of the transducer array. The sub-apertures may be one-dimensional lines of transducers or two-dimensional patches of transducers in a larger array. In transmission mode, the focus and steering of the ultrasonic beam generated by the array or sub-apertures of the array are controlled as described below.

[0123] When receiving a backscattered echo signal from the subject, the received signal undergoes beamforming (as described later) to align it, and if a sub-aperture is used, the sub-aperture is shifted by, for example, one transducer element. The shifted sub-aperture is then activated, and the process is repeated until all transducer elements in the transducer array are activated.

[0124] For each line (or sub-aperture), the total received signal used to form the relevant line in the final ultrasound image is the sum of the voltage signals measured by the transducer element of a given sub-aperture during the reception period. The resulting line signal is then subjected to the following beamforming process and is typically called radio frequency (RF) data. Each line signal (RF data set) generated by the various sub-apertures then undergoes additional processing to generate the lines in the final ultrasound image. The change in amplitude of the line signal over time contributes to the change in brightness of the ultrasound image with depth, and high amplitude peaks correspond to bright pixels (or clusters of pixels) in the final image. Peaks appearing near the beginning of the line signal represent echoes from shallow structures, while peaks that gradually appear after the line signal represent echoes from structures at greater depths within the subject.

[0125] One of the functions controlled by the transducer controller 118 is the direction in which the beam is steered and focused. The beam may be steered straight forward (orthogonally) from the transducer array, or at different angles relative to a wider field of view. The steering and focusing of the transmit beam can be controlled as a function of the operating time of the transducer elements.

[0126] Two methods, namely plane-wave imaging and "beam-steering" imaging, can be distinguished in general ultrasound data acquisition. The two methods are distinguished by the presence of beamforming in the transmission ("beam-steering" imaging) and / or reception modes (plane-wave imaging and "beam-steering" imaging).

[0127] First, looking at the focusing function, by operating all transducer elements simultaneously, the transducer array generates a plane wave that diverges as it travels through the subject. In this case, the ultrasound beam remains unfocused. By introducing a position-dependent time delay in the activation of the transducers, it is possible to focus the wavefront of the beam to a desired point called the focal zone. The focal zone is defined as the point where the lateral beamwidth is less than half the transmitted beamwidth. In this way, the lateral resolution of the final ultrasound image is improved.

[0128] For example, if the time delay activates the transducer elements sequentially, starting from the outermost elements of the transducer array and finishing with the central elements, the focal zone will be formed along the central elements at a predetermined distance from the probe. The distance of the focal zone from the probe varies depending on the time delay between each subsequent round of transducer element activation. After the beam passes through the focal zone, it begins to diverge, forming the far-field imaging region. Note that in focal zones located close to the transducer array, the ultrasonic beam diverges rapidly in the far field, leading to beam width artifacts in the final image. Typically, the near field located between the transducer array and the focal zone shows little detail due to the large overlap of the ultrasonic beam. Therefore, changing the position of the focal zone can result in significant changes in the quality of the final image.

[0129] Note that in transmission mode, only one focus can be defined unless the ultrasound image is divided into multiple focal zones (each potentially having a different transmission focus).

[0130] Furthermore, upon receiving an echo signal from within the subject, the reverse of the above process can be performed to perform receive focusing. In other words, the input signal can undergo an electron time delay before being received by the transducer element and passed to the system for signal processing. The simplest example of this is called delayed-sum beamforming. It is possible to dynamically adjust the receive focus of the transducer array as a function of time.

[0131] Looking at the beam steering function here, it is possible to give the ultrasonic beam a desired angle as it leaves the transducer array through the correct application of time delays to the transducer elements. For example, by activating a transducer on the first side of the transducer array and then activating the remaining transducers in a sequence ending on the opposite side of the array, the wavefront of the beam is angled toward the second side. The magnitude of the steering angle relative to the normal of the transducer array depends on the magnitude of the time delay between the activations of the subsequent transducer elements.

[0132] Furthermore, the total time delay applied to each transducer element can be the sum of the focusing time delay and the steering time delay, allowing the steering beam to be focused. In this case, the transducer array is called a phased array.

[0133] For CMUT transducers that require a DC bias voltage for activation, the transducer controller 118 can be coupled to control a DC bias control 145 for the transducer array. The DC bias control 145 sets the DC bias voltage applied to the CMUT transducer elements.

[0134] For each transducer element in the transducer array, an analog ultrasonic signal, typically called channel data, enters the system via a receiving channel. In the receiving channel, a partially beamformed signal is generated from the channel data by a microbeamformer 112 and then passed to the main receiving beamformer 120, where the partially beamformed signals from individual patches of transducers are combined into a fully beamformed signal called radio frequency (RF) data. The beamforming performed at each stage may be as described above or may include additional functions. For example, the main beamformer 120 may have 128 channels, each receiving partially beamformed signals from patches of tens or hundreds of transducer elements. In this way, signals received by thousands of transducers in the transducer array can efficiently contribute to a single beamformed signal.

[0135] The beamformed received signal is coupled to the signal processor 122. The signal processor 122 can process the received echo signal in various ways, including band-pass filtering, decimation, I and Q component separation, and harmonic signal separation, which acts to separate linear and nonlinear signals to enable identification of nonlinear (fundamental frequency harmonics) echo signals returned from tissue and microbubbles. The signal processor can also perform additional signal enhancements such as speckle reduction, signal synthesis, and noise reduction. The bandpass filter within the signal processor can be a tracking filter, whose passband slides from higher frequency bands to lower frequency bands as the echo signal is received from increasing depths, thereby eliminating noise at higher frequencies from deeper depths, which typically lack anatomical information.

[0136] Beamformers for transmission and reception can be implemented with different hardware and have different functions. Of course, receiver beamformers are designed to take into account the characteristics of the transmission beamformer. In Figure 5, for simplification, only receiver beamformers 112 and 120 are shown. In a complete system, there is also a transmission chain that includes a transmission microbeamformer and a main transmission beamformer.

[0137] The function of the microbeam former 112 is to provide an initial combination of signals to reduce the number of analog signal paths. This is typically performed in the analog domain.

[0138] The final beamforming is performed in the main beamformer 120, typically after digitization.

[0139] The transmit and receive channels use the same transducer array 106 having a fixed frequency bandwidth. However, the bandwidth occupied by the transmit pulse may vary depending on the transmit beamforming used. The receive channel can capture the entire transducer bandwidth (classical approach), or, by using band-pass processing, only the bandwidth containing the desired information (e.g., harmonics of the principal harmonics) can be extracted.

[0140] The RE signal can then be coupled to the B-mode (i.e., luminance mode, or 2D imaging mode) processor 126 and the Doppler processor 128. The B-mode processor 126 performs amplitude detection on the received ultrasound signal for imaging of internal structures such as organs, tissues, and blood vessels. In line-by-line imaging, each line (beam) is used to generate a luminance value to which the associated RE signal amplitude should be assigned to a pixel in the B-mode image. The precise location of the pixel in the image is determined by the location of the RE signal-related amplitude measurement and the number of lines (beams) of the RF signal. B-mode images of such structures can be formed in harmonic or fundamental image modes, or a combination of both, as described in U.S. Patent 6,283,919 (Roundhill et al.) and U.S. Patent 6,458,083 (Jago et al.). The Doppler processor 128 processes temporally different signals resulting from tissue movement and blood flow for the detection of moving substances such as blood cell flow in the image field. The Doppler processor 128 typically includes a wall filter with parameters set to allow or reject echoes returned from selected types of material in the body.

[0141] The structural and motion signals generated by the B-mode and Doppler processors are coupled to the scan converter 132 and the multiplanar reformatter 144. The scan converter 132 arranges the echo signals in the desired image format in the spatial relationship from which they are received. In other words, the scan converter acts to convert the RF data from a cylindrical coordinate system to a Cartesian coordinate system suitable for displaying ultrasound images on the image display 140. For B-mode images, the brightness of a pixel at a given coordinate is proportional to the amplitude of the RF signal received from that location. For example, the scan converter can arrange the echo signals in a two-dimensional (2D) sector format or a pyramidal three-dimensional (3D) image. The scan converter can superimpose a B-mode structural image having a color corresponding to the motion at a point in the image field, where the Doppler estimated velocity generates a given color. The combined B-mode structural image and color Doppler image depict the movement of tissue and blood flow within the structural image field. As described in U.S. Patent 6,443,896 (Detmer), a multiplanar reformatter converts echoes received from a point in a common plane within a volume region of the body into an ultrasound image of that plane. As described in U.S. Patent 6,530,885 (Entrekin et al.), a volume renderer 142 converts echo signals from a 3D dataset into a projected 3D image viewed from a given reference point.

[0142] 2D or 3D images are coupled from the scan converter 132, multiplanar reformatter 144, and volume renderer 142 to the image processing device 130, where they undergo further expansion, buffering, and temporary storage for optional display on the image display 140. The imaging processor may be adapted to remove specific imaging artifacts from the final ultrasound image, such as acoustic shadowing caused by strong attenuation or refraction, back-weighting caused by weak attenuation, or reverberation artifacts where highly reflective tissue interfaces are located in close proximity. In addition, the image processor may be adapted to process specific speckle reduction functions to improve the contrast of the final ultrasound image.

[0143] In addition to being used for imaging, blood flow values ​​generated by the Doppler processor 128 and tissue structure information generated by the B-mode processor 126 are coupled to the quantification processor 134. The quantification processor generates measurements of different flow conditions, such as blood flow volume velocity, in addition to structural measurements such as organ size and gestational age. The quantification processor can receive input from the user control panel 138, such as points within the anatomical structures of the image on which the measurements are being taken.

[0144] The output data from the quantification processor is coupled to the graphics processor 136 to reproduce measurement graphics and values ​​using images on the display 140 and to output audio from the display device 140. The graphics processor 136 can also generate graphic overlays for display with the ultrasound images. These graphic overlays may include standard identification information such as patient name, date and time of image, and imaging parameters. For these purposes, the graphics processor receives input such as patient name from the user interface 138. The user interface is also coupled to the transmit controller 118 to control the generation of ultrasound signals from the transducer array 106, and therefore the generation of images generated by the transducer array and ultrasound system. The transmit control function of the controller 118 is only one of the functions it performs. The controller 118 also takes into account the operating mode (given by the user) and the corresponding required transmitter and bandpass settings in the receiver analog / AD converter. The controller 118 can be a state machine with fixed states.

[0145] The user interface is also coupled to a multiplanar formatter 144 for the selection and control of planes in multiple multiplanar format (MPR) images, which can be used to perform quantified measurements in the image field of the MPR images.

[0146] The ultrasonic system described above can be operably coupled with the processing device 32 described above. For example, the ultrasonic system described above can be used in some examples to implement the ultrasonic acquisition device 54 of the system 30 shown in Figure 2.

[0147] The embodiments of the present invention described above utilize a processing apparatus. The processing apparatus may generally comprise a single processor or multiple processors. It may be located within a single housing, structure, or unit, or it may be distributed across multiple different devices, structures, or units. Therefore, a reference to a processing apparatus adapted or configured to perform a particular step or task may correspond to that step or task being performed by any one or more of the multiple processing components, individually or in combination. Those skilled in the art will understand how such a distributed processing apparatus can be implemented. The processing apparatus includes a communication module or input / output for receiving data and outputting data to further components.

[0148] One or more processors in a processing unit can be implemented in numerous ways using software and / or hardware to perform various required functions. A processor typically uses one or more microprocessors that can be programmed using software (e.g., microcode) to perform the required functions. A processor can be implemented as a combination of dedicated hardware for performing some functions and one or more programmed microprocessors and associated circuitry for performing other functions.

[0149] Examples of circuits that may be used in various embodiments of this disclosure include, but are not limited to, conventional microprocessors, application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs).

[0150] In various implementations, a processor may be associated with one or more storage media, such as volatile and non-volatile computer memory, including RAM, PROM, EPROM, and EEPROM. These storage media may be encoded with one or more programs that, when running on one or more processors and / or controllers, perform the necessary functions. Various storage media may be fixed within a processor or controller, or they may be portable, so that the one or more programs stored on them can be loaded into the processor.

[0151] Variations of the disclosed embodiments can be understood and implemented by those skilled in the art in carrying out the claimed invention, based on the study of the drawings, disclosures, and appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude the plural.

[0152] A single processor or other unit can fulfill the functions of several items listed in the claims.

[0153] The mere fact that certain means are described in different dependent claims does not mean that combinations of these means cannot be used advantageously.

[0154] Computer programs may be stored / distributed on suitable media such as optical storage media or solid-state media supplied together with or as part of other hardware, but they may also be distributed in other forms such as the Internet or other wired or wireless telecommunications systems.

[0155] Please note that when the term "conformed to" is used in the claims or description, it is intended to be equivalent to the term "configured to".

[0156] No reference numeral in a claim should be construed as limiting the scope.

Claims

1. A method for operating a processing device for an ultrasonic acquisition device, wherein the processing device has a processor, and the method is The processor receives real-time 3D ultrasound data of an anatomical object from the ultrasound acquisition device, comprising a series of one or more 3D frames, wherein each 3D frame spans a 3D volume. The processor determines a plurality of target planes through the 3D volume to be extracted and records the target planes in a target plane list. The processor performs a planar capture operation, wherein for each of the 3D frames of the 3D ultrasound data, The processor includes the steps of extracting a selected set of the determined target planes from the 3D ultrasound data, The processor controls the user interface to provide a real-time ultrasound image display output that shows a selected subset of the extracted set of planes of the current frame. The processor stores the extracted plane in a plane dataset, and Steps having, The processor performs a quality evaluation operation in parallel with the planar capture operation, The processor applies a predetermined quality evaluation algorithm to the planes in the planar dataset to determine, for each target plane in the target plane list, whether any individual extracted plane in the planar dataset corresponding to the target plane satisfies a predetermined quality standard, or whether the time series of extracted planes in the planar dataset corresponding to the same target plane satisfies a predetermined quality standard. The processor updates the target plane list and removes any of the target planes that satisfy the predefined quality standards from the list. Steps having, The processor repeats the steps of the plane capture operation for each subsequent 3D frame in the sequence of frames until the target plane list is empty. It has, For each frame, the selected set of target planes to be extracted has at least each of the planes still included in the target plane list. For each frame, the selected subset of extracted planes to be displayed on the user interface includes at least one of the planes still included in the target plane list. method.

2. The step of the processor determining a plurality of target planes is: The processor includes the steps of applying an anatomical segmentation algorithm to at least one of the 3D frames to identify one or more predefined anatomical locations within the 3D frame, The processor searches a predefined list of target views, each target view being associated with an anatomical location. The processor performs the steps of determining the target plane based on the target view and the segmentation of the 3D frame. The method according to claim 1, comprising:

3. The method according to claim 1, further comprising the step of the processor displaying a visual indication on the user interface of the target planes remaining on the target plane list.

4. The method according to claim 3, wherein the visual indication comprises a list of texts and / or a graphical representation of the target plane.

5. The method according to claim 1, wherein the selected subset of the extracted plurality of planes displayed on the user interface consists of only one plane from the extracted plurality of planes.

6. The user interface includes a first display area and a second display area, The aforementioned method, The processor controls the user interface to display a selected subset of the plurality of extracted planes in the first display area with a first image size, The processor controls the user interface to display the plurality of extracted planes, each having an image size smaller than the first image size, in the second display area. The method according to claim 1, having the following characteristics.

7. The method according to claim 6, further comprising the step of the processor controlling the user interface to display in the second display area all of the extracted planes of the current frame, each annotated with a visual indication of whether the plane remains on the target plane list.

8. The method according to claim 7, wherein the visual indication has a boundary around the displayed extracted plane, and the method comprises the step of the processor modifying the graphical or visual properties of the boundary depending on whether the extracted plane is still on the target plane list.

9. The method according to claim 1, wherein the quality classification has graded quality scores, or the quality classification is a binary pass / fail classification.

10. The method according to claim 1, wherein the processor provides user control that controls the user interface to enable the user to invalidate the quality classification output from the quality evaluation for each extracted plane or time series of planes, thereby enabling the user to manually add or remove any extracted plane from the target plane list.

11. The quality evaluation algorithm is configured to identify extracted planes or time series of planes that pass or fail the quality standard by a margin smaller than a predefined threshold margin, and to classify the extracted planes using boundary classification. The method includes the step of the processor controlling the user interface to display a visual indication of the plane using boundary classification and prompting the user for user input. The aforementioned quality classification is fixed as either pass or fail depending on the user input. The method according to claim 1.

12. A computer program product having computer program code configured to cause the processor to perform the method described in any one of claims 1 to 11 when executed on the processor.

13. A processing device for use with an ultrasound acquisition device during an ultrasound scan performed by a user, comprising input / output, One or more processors, The steps include receiving real-time 3D ultrasound data of an anatomical object having a series of 3D frames from the ultrasound acquisition device, wherein each 3D frame spans a 3D volume, and The steps include determining multiple target planes through the 3D volume to be extracted and recording the target planes in a target plane list, A step of performing a planar capture operation, wherein for each received frame of the 3D ultrasound data, The steps include extracting a selected set of the determined target planes from the 3D ultrasound data, The steps include controlling the user interface to provide a real-time ultrasound image display output that shows a selected subset of the extracted set of planes of the current frame, The steps include storing the extracted planes in a plane dataset and Steps having, A step in which a quality evaluation operation is performed in parallel with the aforementioned planar capture operation, The steps include: applying a predetermined quality evaluation algorithm to the planes in the plane dataset to determine, for each target plane in the target plane list, whether any of the individual extracted planes in the plane dataset corresponding to the target plane satisfy a predetermined quality standard, or whether the time series of extracted planes in the plane dataset corresponding to the same target plane satisfy a predetermined quality standard; The steps include updating the target plane list and removing any of the target planes that meet the predefined quality standards from the list. Steps having, The steps of the plane capture operation are repeated for each subsequent 3D frame in the sequence of frames until the target plane list is empty. It has, For each frame, the selected set of extracted target planes has at least each of the planes still included in the list. For each frame, the selected subset of extracted planes to be displayed on the user interface includes one or more processors, each including at least one of the planes still included in the list. A processing apparatus having

14. The apparatus according to claim 13, An ultrasonic acquisition system equipped with an ultrasonic probe for user operation and A system that includes these features.

15. The system according to claim 14, further comprising a user interface having a display unit.

Images